Unmanned aerial vehicle inspection method and device for tower component of power distribution network and unmanned aerial vehicle

Abstract

The invention provides an unmanned aerial vehicle inspection method and device for tower components of a power distribution network and an unmanned aerial vehicle, and belongs to the field of power inspection. The method comprises the following steps: acquiring an image set acquired in real time, and determining identification information of a detected object on the image set based on the image set; determining a first position of the unmanned aerial vehicle based on the positions and sizes of the detection objects in the visible light image and the standard image; acquiring first environment data and a first multispectral image; the first multispectral image is shot by the unmanned aerial vehicle at a first position, and the first environment data is data of an environment where the unmanned aerial vehicle is located when shooting the first multispectral image; based on the first environment data and environment data during shooting of the standard multispectral image, correcting the spectral reflectivity of the detected object in the first multispectral image to obtain a first spectral reflectivity curve of the detected object; and detecting the detected object based on the first spectral reflectivity curve and the standard spectral reflectivity curve. The accuracy of unmanned aerial vehicle inspection can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power inspection, and in particular to a method and device for inspecting pole tower components of a power distribution network using an unmanned aerial vehicle (UAV), and an unmanned aerial vehicle (UAV). Background Art

[0002] With the development of science and technology, drones are increasingly used in the power industry, especially in the field of distribution network inspection. Drones can complete a large number of tower inspections in a short period of time, improve work safety, and greatly reduce the time and labor costs required for inspections.

[0003] The drone can accurately identify the distribution network tower and its components by carrying a gimbal camera. This includes identifying important components such as the tower body, conductors, insulators, and grounding wires to ensure that they are in normal operation. It can also detect abnormalities in the tower and its ancillary facilities, such as broken insulators, rusted metal parts, and foreign object interference. These abnormalities may cause power grid failures or safety hazards. Early detection and early warning will help avoid accidents.

[0004] At present, the pole tower components of the distribution network are mainly obtained through visible light images by cameras, and the target detection algorithm is used for component identification. However, the component recognition algorithm under the visible light image may be sensitive to factors such as lighting, angle, and occlusion, and it is easy to make misjudgments or miss detections. In addition, the distribution network pole tower components are very diverse, and there are the same components with different models and different states. Visible light images alone may not provide enough information to distinguish the model and state. Summary of the invention

[0005] The embodiments of the present invention provide a method, a device and a drone for drone inspection of pole tower components of a distribution network, so as to solve the problem of low accuracy of current recognition models.

[0006] In a first aspect, an embodiment of the present invention provides a method for inspecting a tower component of a power distribution network by using a drone, comprising:

[0007] Acquire an image set collected in real time, and determine identification information of the detection object on the image set based on the image set; wherein the image set includes a visible light image and a multispectral image of the detection object;

[0008] Determine a first position of the drone based on the position and size of the detected object in the visible light image and the standard image; wherein the standard image is determined from a standard image library based on identification information of the detected object;

[0009] Acquire first environmental data and a first multispectral image; the first multispectral image is taken by the drone at the first position, and the first environmental data is data of the environment in which the drone is located when taking the first multispectral image;

[0010] Based on the first environmental data and the environmental data when the standard multispectral image is taken, the spectral reflectance of the detection object in the first multispectral image is corrected to obtain a first spectral reflectance curve of the detection object;

[0011] The detection object is detected based on the first spectral reflectance curve and the standard spectral reflectance curve.

[0012] In a possible implementation, determining identification information of a detection object in the image set based on the image set includes:

[0013] Input the multispectral image into the multispectral image detection module to obtain the category of the detected object;

[0014] Based on the category of the detected object, the visible light image, and the category boundary detection frame template set, a boundary detection frame of the detected object on the visible light image is determined.

[0015] In a possible implementation, determining the first position of the drone based on the position and size of the detected object in the visible light image and the standard image includes:

[0016] Based on the category of the detected object, a boundary detection frame template corresponding to the detected object is selected from the category boundary detection frame template set;

[0017] Based on the boundary detection frame template corresponding to the detection object, determining the boundary detection frame of the detection object in the standard image containing the detection object;

[0018] The first position of the drone is determined based on the positions of the boundary detection frame on the visible light image and the boundary detection frame in the standard image, and the sizes of the detected objects in the boundary detection frame of the visible light image and the boundary detection frame in the standard image.

[0019] In a possible implementation, determining the first position of the drone based on the positions of the boundary detection frame on the visible light image and the boundary detection frame in the standard image, and the size of the detected object in the boundary detection frame of the visible light image and the size of the detected object in the boundary detection frame of the standard image, includes:

[0020] Based on the size of the detection object in the boundary detection frame of the visible light image and the size of the detection object in the boundary detection frame of the standard image, respectively, determining the first distance that the drone needs to adjust; wherein the first distance is the vertical distance between the drone and the center point of the detection object, the size of the detection object in the boundary detection frame on the second visible light image is equal to the size of the detection object in the boundary detection frame in the standard image, and the second visible light image is obtained by the drone after adjusting the first distance;

[0021] Based on the position of the boundary detection frame on the visible light image and the boundary detection frame in the standard image, determine the second distance that the drone needs to adjust; wherein the second distance is the distance from the center point of the camera on the drone projected onto the surface of the detection object to the center point of the detection object, the position of the boundary detection frame on the third visible light image in the third visible light image is the same as the position of the boundary detection frame in the standard image in the standard image, the third visible light image and the standard image have the same size, and the third visible light image is acquired by the drone after adjusting the second distance;

[0022] Based on the first distance and the second distance, a first position of the drone is determined.

[0023] In a possible implementation, the environmental data includes temperature and light, the first environmental data includes a first temperature and a first light, and the standard environmental data includes a standard temperature and a standard light;

[0024] Based on the first environment data and the standard environment data when the standard multispectral image is captured, the spectral reflectance of the detection object in the first multispectral image is corrected, including:

[0025] Determining a first variation factor based on the first temperature, the standard temperature, and the preset temperature spectrum variation curve;

[0026] Determining a second variation factor based on the first illumination, the standard illumination, and a preset illumination spectrum variation curve;

[0027] The spectral reflectance of the detection object in the first multispectral image is corrected based on the first variation factor and the second variation factor.

[0028] In a possible implementation, correcting the spectral reflectance of the detection object in the first multispectral image based on the first variation factor and the second variation factor includes:

[0029] Inputting the first change factor and the second change factor into a reflectivity correction model to obtain a third change factor; wherein the reflectivity correction model is a neural network model, which is trained based on historical temperature changes and historical light changes;

[0030] The spectral reflectance of the detection object in the first multispectral image is corrected based on the third variation factor.

[0031] In a possible implementation, the standard multispectral image and the standard image are images taken at the same location and containing the detection object, and the standard spectral reflectance curve is the spectral reflectance curve of the detection object in the standard multispectral image;

[0032] Detecting the object to be detected based on the first spectral reflectance curve and the standard spectral reflectance curve, including:

[0033] When the difference between the absolute value of the spectral reflectance of the target wavelength in the first spectral reflectance curve and the absolute value of the spectral reflectance in the standard spectral reflectance curve is greater than a preset difference threshold, it is determined that the detected object is abnormal; wherein the target wavelength is any wavelength in the spectral reflectance curve;

[0034] When the sum of the absolute values ​​of the spectral reflectances of all target wavelengths in the first spectral reflectance curve and the absolute values ​​of the spectral reflectances in the standard spectral reflectance curve is greater than a first preset sum threshold, or is less than a second preset sum threshold, it is determined that the detected object is abnormal; wherein the first preset sum threshold is greater than the second preset sum threshold.

[0035] In a possible implementation, the method further includes:

[0036] When an abnormality occurs in the detection result, the first multispectral image, the standard multispectral image, the first spectral reflectance curve and the standard spectral reflectance curve are transmitted back to the central server.

[0037] In a second aspect, an embodiment of the present invention provides a drone inspection device for tower components of a distribution network, comprising:

[0038] A first acquisition module is used to acquire an image set collected in real time, and determine identification information of the detection object on the image set based on the image set; wherein the image set includes a visible light image and a multispectral image of the detection object;

[0039] A determination module, configured to determine a first position of the drone based on the position and size of the detection object in the visible light image and the standard image; wherein the standard image is determined from a standard image library based on identification information of the detection object;

[0040] The second acquisition module is used to acquire first environmental data and a first multispectral image; the first multispectral image is taken by the drone at the first position, and the first environmental data is data of the environment in which the drone is located when taking the first multispectral image;

[0041] A processing module, configured to correct the spectral reflectance of the detection object in the first multispectral image based on the first environmental data and the environmental data when the standard multispectral image is taken, so as to obtain a first spectral reflectance curve of the detection object;

[0042] The detection module is used to detect the detection object based on the first spectral reflectance curve and the standard spectral reflectance curve.

[0043] In a third aspect, an embodiment of the present invention provides a drone, comprising a drone inspection device for pole tower components of a distribution network after knowledge distillation and lightweight processing, wherein the drone inspection device for pole tower components of a distribution network is deployed in a drone, and the drone inspection device for pole tower components of a distribution network is used to perform inspections based on any method of the first aspect.

[0044] The embodiment of the present invention provides a method, device and drone for unmanned aerial vehicle inspection of pole tower components of a distribution network. First, by acquiring an image set collected by a drone in real time, and determining the identification information of the detection object on the image set based on the image set, the category of the detection object is determined, so as to facilitate further detection of the detection object. Next, in order to improve the accuracy of detection, it is also necessary to determine the detection position of the drone, and the first position of the drone can be determined by the position and size of the detection object in the visible light image and the standard image. The multispectral image taken by the drone at the first position is the same as the position of the standard multispectral image. Then, the first environmental data and the first multispectral image are obtained at the first position. Since the environment has a greater influence on the spectrum, it is also necessary to correct the spectral reflectance of the detection object in the first multispectral image based on the first environmental data and the environmental data when the standard multispectral image is taken, and obtain the first spectral reflectance curve of the detection object. Finally, based on the first spectral reflectance curve and the standard spectral reflectance curve, the detection object is detected. Thus, not only the detection accuracy of the drone inspection can be improved, but also the detection efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 It is a flow chart of an implementation method of a drone inspection method for tower components of a distribution network provided by an embodiment of the present invention;

[0047] Figure 2 is a schematic diagram of the structure of a twin model provided by an embodiment of the present invention;

[0048] Figure 3 It is a structural schematic diagram of an unmanned aerial vehicle inspection device for tower components of a power distribution network provided by an embodiment of the present invention;

[0049] Figure 4 Schematic diagram of the structure of the UAV provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0050] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0052] Figure 1 The implementation flow chart of the drone inspection method for the tower components of the distribution network provided by the embodiment of the present invention is detailed as follows:

[0053] S110, acquiring an image set collected in real time, and determining identification information of the detection object in the image set based on the image set.

[0054] The drone is equipped with a multispectral camera, which can collect visible light images and multispectral images of the detected object in real time.

[0055] In addition, the drone is also equipped with an inspection device that uses knowledge distillation and lightweight processing. This device can reduce the number of parameters for component identification and anomaly detection through knowledge distillation. It can also process the collected images in real time during the flight and judge whether the state of the detected object is abnormal in real time. This not only saves computing resources and improves the reasoning speed, but also meets the limited performance requirements of the edge module.

[0056] In some embodiments, the lightweight multispectral image detection module and processing device can be deployed to the drone through hardware and software platforms such as NVIDIA Jetson and TensorRT. Then, the acquired images or videos are accessed through DeepStream to reduce the performance loss caused by video decoding, video encoding and streaming media transmission. And the video is converted into an image.

[0057] In this embodiment, the original component recognition and anomaly detection methods can be optimized and streamlined by knowledge distillation to reduce the number of model parameters. Knowledge distillation technology transfers knowledge between the auxiliary model and the target model, allowing the target model to learn the feature representation, category distribution and other information of the auxiliary model, thereby improving the generalization ability and reasoning speed of the model.

[0058] Then, the knowledge distilled method is quantized and pruned. Specifically, first, the parameters are converted from high-precision floating point numbers to low-precision floating point numbers. Then, redundant neurons are identified and removed through pruning. The combination of quantization and pruning can effectively reduce the size, computational workload, and memory usage of the model, and improve the inference speed to better adapt to the resource constraints of edge devices.

[0059] Finally, by connecting video streams of different spectra through DeepStream, fast video stream decoding, encoding and transmission can be achieved, while having efficient streaming media processing capabilities. Real-time analysis of streaming media through DeepStream can effectively reduce performance loss, ensure smooth and efficient data processing, and provide reliable video data support for subsequent component identification and anomaly detection. And it can efficiently and quickly convert videos into images.

[0060] In some embodiments, a multispectral image detection module is provided in the drone through knowledge distillation and lightweight processing. After obtaining a multispectral image of the detection object, the multispectral image can be input into the multispectral image detection module to obtain the category of the detection object.

[0061] The objects to be tested can be the tower body, conductors, insulators, ground wires and other tower components of the distribution network tower.

[0062] In this embodiment, the multispectral image detection module includes a backbone network unit, a multispectral channel attention mechanism unit and a detection and positioning unit. The backbone network unit is used to obtain features under different spectra and perform feature fusion according to the channel dimension in the neck network. By introducing the channel attention mechanism unit, the importance of each channel of the feature map is obtained, and each feature is assigned a corresponding weight according to the importance, so that the network focuses on certain feature channels. The channels of the feature map that are useful for the component recognition task are enhanced, and the feature channels that are not very useful for the current task are suppressed. Finally, the acquired features are detected and positioned through the detection and positioning unit.

[0063] Specifically, the backbone network unit can use the Darknet53 network, and the steps of feature extraction are as follows:

[0064] First, CSPDarknet53 performs neighboring downsampling before extracting features to generate four complementary but different sub-images, converting width and height information into channel dimension information. The four images are concat-operated to obtain a new image with 12 channels, and then the Conv operation is performed to obtain a double-downsampled feature map with no information loss. Assuming the image shape is Nx3xHxW, the slicing operation is used to convert it into a feature map of Nx12xH / 2xW / 2, and then the Conv operation is used to convert it into a feature map of Nx32xH / 2xW / 2.

[0065] Next, the C3 module is a key component of CSPDarknet53, which contains 3 standard convolutional layers and multiple BottleNeck modules. The BottleNeck module draws on the residual structure of ResNet, where the left branch halves the number of channels of the feature map through Conv1x1, and then doubles the number of channels through Conv3x3 convolution to extract features. The right branch is connected to the residual through shortcut and added to the output features of the left branch. Finally, while effectively solving the gradient problem, 3 features of different scales are extracted.

[0066] Specifically, the processing steps of the multi-spectral channel attention mechanism unit are as follows:

[0067] First, Concat multi-spectral features are fused.

[0068] In the neck network, the feature maps of different spectra are concatenated according to the channel dimension to achieve the fusion of multi-spectral features. The information under different spectra can be integrated by concatenating the feature maps, so as to fully consider the profile, material and temperature of the tower components.

[0069] Second, multi-spectral channel attention mechanism.

[0070] The feature map of multi-spectral splicing is processed according to the Channel Attention Mechanism (CAM), which is divided into two parts: compression and excitation. The purpose of the compression part is to compress the information in the global space, and then perform feature learning in the channel dimension to form the importance of each channel to the channel. Finally, different weights are assigned to each channel through the excitation part.

[0071] In the compression part, the squeeze operation is used to compress the feature map into a feature vector through global average pooling. The input feature map dimension is H×W×C, which represents the height, width, and number of channels respectively. Compression compresses the feature map dimension from H×W×C to 1×1×C.

[0072] In the excitation part, the excitation operation uses a fully connected layer to reduce the number of channels of the feature map to 1 / r of the original. Then, after passing through the Swish function and the fully connected layer, the channel dimension of the feature map is increased to 1×1×C. Finally, it is converted into a normalized weight vector between 0 and 1 through the sigmoid function. Through the scale operation, the normalized weight is multiplied by the original input feature map channel by channel to generate a weighted feature map.

[0073] The processing steps for detecting the positioning unit are as follows:

[0074] The main part of the detection head is 3 Detect detectors, whose network structure consists of only three Conv1x1, corresponding to three detection feature layers. It uses grid-based anchors to detect targets on feature maps of different scales. The main process is: after obtaining the output, compare it with the real data annotation, calculate the positioning loss, classification loss and confidence loss, and then reshape the data format as needed, and perform corresponding activation positioning on the original grid coordinates.

[0075] In some embodiments, in order to accurately locate the position of the multispectral camera on the drone from the detected object, it is also necessary to determine the boundary detection frame of the detected object on the visible light image based on the category of the detected object, the visible light image, and the category boundary detection frame template set. At this time, the boundary detection frame of the detected object can be displayed on the visible light image.

[0076] The category boundary detection frame template set includes a boundary detection frame template corresponding to each detection object to be detected, and the boundary detection frame template can completely place the detection object in the frame.

[0077] The sizes of the visible light image and the multispectral image captured by the multispectral camera in the present invention are the same, and the visible light image and the multispectral image collected in the present invention are both for a single detection object.

[0078] S120: Determine a first position of the drone based on the position and size of the detected object in the visible light image and the standard image.

[0079] Among them, the standard image is determined from the standard image library based on the identification information of the detection object. The standard image library stores standard images of all detection objects to be detected. The standard image is taken based on a specific shooting position and specific environmental conditions. It should be noted that. Each detection object corresponds to a standard image, and the standard image of each detection object is taken in advance. The environmental conditions here are temperature and light. The size of the standard image and the visible light image is also the same. Each detection object corresponds to only one standard image, and each standard image contains only one detection object. When determining the first position of the drone, it is also determined based on the visible light image of the same detection object and the position and size of the detection object in the standard image.

[0080] When shooting visible light images or multispectral images, the shooting distance and shooting angle are different, which may cause the captured images to be shifted, stretched, scaled, etc., which will have a great impact on the subsequent inspection results and reduce the accuracy of the detection. In order to improve the accuracy of the inspection results, the shooting position of the drone needs to be determined.

[0081] In some embodiments, after the category of the detected object is determined, first, based on the category of the detected object, a boundary detection frame template corresponding to the detected object can be screened out from a category boundary detection frame template set.

[0082] Then, based on the boundary detection frame template corresponding to the detection object, a boundary detection frame of the detection object in the standard image containing the detection object is determined.

[0083] Finally, based on the positions of the boundary detection frame on the visible light image and the boundary detection frame in the standard image, and the sizes of the detected objects in the boundary detection frame of the visible light image and the boundary detection frame in the standard image, the first position of the drone is determined.

[0084] In this embodiment, the vertical distance between the drone and the detection object can be adjusted by adjusting the size of the detection object in the boundary detection frame of the visible light image and the size of the boundary detection frame in the standard image, so as to ensure that the newly captured image is in the same vertical plane from the detection object as the standard image. By adjusting the position of the boundary detection frame of the detection object on the visible light image and the boundary detection frame of the detection object in the standard image, the shooting position of the multispectral camera on the drone can be made the same as the shooting position of the standard image.

[0085] Specifically, based on the size of the detected object in the boundary detection frame of the visible light image and the size of the detected object in the boundary detection frame of the standard image, the first distance that the drone needs to adjust is determined, wherein the first distance is the vertical distance between the drone and the center point of the detected object.

[0086] When the vertical distance between the drone and the center point of the detection object is different, the size of the detection object photographed will be different, and thus the size in the boundary detection frame will also be different. In order to ensure that the drone's shooting position is fixed, the first distance that the drone needs to adjust can be confirmed first. After the drone adjusts based on the first distance, the second visible light image is captured. The size of the detection object in the boundary detection frame on the second visible light image is equal to the size in the boundary detection frame in the standard image.

[0087] Based on the position of the boundary detection frame on the visible light image and the boundary detection frame in the standard image, determine the second distance that the drone needs to adjust. The second distance is the horizontal and vertical distance from the center point of the camera on the drone projected onto the surface of the detection object to the center point of the detection object. After the drone makes adjustments based on the second distance, it captures a third visible light image, and the position of the boundary detection frame on the third visible light image in the third visible light image is the same as the position of the boundary detection frame in the standard image in the standard image.

[0088] Finally, based on the first distance and the second distance, a first position of the drone is determined.

[0089] S130: Acquire first environmental data and a first multispectral image.

[0090] The first multispectral image is taken by the drone when it is in the first position. When the drone is in the first position, the gimbal and focal length are automatically adjusted, and during the adjustment, automatic exposure is performed based on the light intensity at the first position to ensure that the exposure of each shot is kept within a certain threshold range.

[0091] The first environmental data is data of the environment in which the drone is located when taking the first multispectral image.

[0092] In some embodiments, the environmental data here include temperature and light. In addition to the shooting distance having a certain impact on the multi-spectral image, the temperature and light during shooting will also have an impact on the multi-spectral image. Therefore, in order to improve the accuracy of the inspection, the impact of temperature and light also needs to be considered.

[0093] S140: Based on the first environment data and the standard environment data when the standard multispectral image is captured, correct the spectral reflectance of the detection object in the first multispectral image to obtain a first spectral reflectance curve of the detection object.

[0094] In some embodiments, the environmental data includes temperature and illumination, the first environmental data includes a first temperature and a first illumination, and the standard environmental data includes a standard temperature and a standard illumination.

[0095] In this embodiment, first, a first variation factor may be determined based on the first temperature, standard temperature, and preset temperature spectrum variation curves. Then, a second variation factor may be determined based on the first illumination, standard illumination, and preset illumination spectrum variation curves. Finally, the spectral reflectance of the detection object in the first multi-spectral image may be corrected based on the first variation factor and the second variation factor.

[0096] Specifically, the preset temperature spectrum change curve is determined based on the historical temperature and the spectral reflectance corresponding to the temperature. The preset illumination spectrum change curve is determined based on the historical illumination and the spectral reflectance corresponding to the temperature.

[0097] Since temperature and light will affect the spectral reflectance at the same time, the spectral reflectance adjusted based on a single factor alone is also inaccurate. In order to further improve the accuracy of the spectral reflectance of the detected object in the first multispectral image, a reflectance correction model can be constructed. And the reflectance correction model is also stored in the drone through distillation and lightweight processing.

[0098] The reflectivity correction model is a neural network model, which is trained based on historical temperature changes and historical light changes. The reflectivity correction model inputs a separate temperature change factor and a light change factor. Since the spectral reflectivity is not based solely on the influence of a single factor of temperature or light, but is affected by both temperature and light, the output third change factor is a factor based on both temperature and light. The reflectivity correction model is based on historical temperature change factors, historical light change factors, and the spectral reflectivity based on the temperature and light at the time. The trained model can accurately predict the third change factor affected by the combined influence of temperature and light.

[0099] Specifically, the first variation factor and the second variation factor may be input into a reflectance correction model to obtain a third variation factor. Then, the spectral reflectance of the detection object in the first multispectral image is corrected based on the third variation factor.

[0100] S150: Detect the detection object based on the first spectral reflectance curve and the standard spectral reflectance curve.

[0101] In some embodiments, detection may be performed based on a reflectivity curve. The standard multispectral image and the standard image are images taken at the same location and containing the detection object, and the standard spectral reflectivity curve is a spectral reflectivity curve of the detection object in the standard multispectral image.

[0102] In this embodiment, when the difference between the absolute value of the spectral reflectance of the target wavelength in the first spectral reflectance curve and the absolute value of the spectral reflectance in the standard spectral reflectance curve is greater than a preset difference threshold, it is determined that the detected object is abnormal. The target wavelength is any wavelength in the spectral reflectance curve. The preset difference threshold is a fixed value determined according to the actual application scenario and the detected object.

[0103] In addition, when the sum of the absolute values ​​of the spectral reflectances of all target wavelengths in the first spectral reflectance curve and the absolute values ​​of the spectral reflectances in the standard spectral reflectance curve is greater than the first preset sum threshold, or less than the second preset sum threshold, it can also be determined that the detected object is abnormal. The first preset sum threshold is greater than the second preset sum threshold, and the first preset sum threshold and the second preset sum threshold are fixed values ​​determined according to the actual application scenario and the detected object.

[0104] The specific abnormal position of the detected object can also be determined based on the difference between the first spectral reflectance curve and the standard spectral reflectance curve for all wavelengths.

[0105] In addition, in some embodiments, when detecting the object to be detected, abnormal detection of components can also be achieved through a twin network. The twin network is a special network structure that can learn the similarity or difference between two inputs. First, the historical image and the current component image are respectively input into the two branches of the twin network to extract features. Then, by calculating the similarity between the features output by the two branches, if the similarity is lower than the set threshold, the current component can be judged as abnormal.

[0106] In this embodiment, if Figure 2 As shown in the figure, the two branches of the Siamese network process the historical normal image and the current component image respectively. The two convolutional networks on the left and right sides have exactly the same structure and share the same weight W. The input data is a set of images (X1, X2, Y), where X1 represents the historical image, X2 represents the current image, Y=0 indicates that component X2 is normal, and Y=1 indicates that component X2 is abnormal. Therefore, the normal pair is (X1, X2+, 0), and the abnormal pair is (X1, X2-, 1). For two different inputs X1 and X2, the high-level feature representations Gw(X1) and Gw(X2) are output respectively. These feature representations capture the key information in the image, such as edges, textures, shapes, etc., and provide useful features for subsequent anomaly detection.

[0107] For a historical normal image X1, if another normal image X2+ of the same component is given, the Euclidean distance between the model's output features Gw(X1) and Gw(X2+) is small, and the loss value becomes larger. If another abnormal photo X2- of the same component is given, the distance between the model's output features Gw(X1) and Gw(X2-) is larger. So D(X1, X2+) <D(X1,X2-)。

[0108] Based on the feature similarity output by the Siamese network, component abnormality judgment is performed. If the similarity between the current component image and the historical component image is lower than the set threshold k, the system will determine that the current component is abnormal and trigger the corresponding abnormal alarm mechanism. The advantage of this method is that it dilutes the label, making the network have good scalability and can classify those categories that have not been trained.

[0109] In some embodiments, the detector may be detected in combination with the spectral reflectance curve and the twin model, which may further improve the detection accuracy.

[0110] In some embodiments, when an abnormality occurs in the detection result, the first multispectral image, the standard multispectral image, the first spectral reflectance curve, and the standard spectral reflectance curve may be compressed and transmitted back to the central server. The central server restores the compressed image through the image recovery service, and further determines the specific abnormal location of the abnormal detection object based on the comparison of the first spectral reflectance curve and the standard spectral reflectance curve, which helps to understand the status and changes of the distribution network tower components and take necessary measures in a timely manner.

[0111] In this embodiment, artificial neural networks can be used for predictive coding, transform coding, vector quantization, etc. based on neural network technology, combined with lossy and lossless compression algorithms, to retain complete image information as much as possible and seek a high compression ratio.

[0112] In addition, deep neural networks have a large number of layers and parameters, and have extremely high requirements for memory and computing power when performing tasks. Deep neural networks cannot meet the needs of drones in terms of memory, energy consumption, and bandwidth. Therefore, it is necessary to use model lightweight technology for image compression models, combined with pruning, weight sharing, and quantization, to reduce the number of model parameters and size, so that it can be applied to conventional edge devices.

[0113] After compressing the image, it is necessary to restore and decode the image. Image compression technology can reduce the cost of data transmission and storage. The compressed image data can be transmitted back to the central server more quickly and effectively for further analysis and recording, which helps to timely understand the status and changes of distribution network tower components.

[0114] The unmanned aerial vehicle inspection method, device and unmanned aerial vehicle provided by the present invention, first, by obtaining an image set collected by the unmanned aerial vehicle in real time, and determining the identification information of the detection object on the image set based on the image set, thereby determining the category of the detection object, so as to facilitate further detection of the detection object. Then, in order to improve the accuracy of detection, it is also necessary to determine the detection position of the unmanned aerial vehicle, and the first position of the unmanned aerial vehicle can be determined by the position and size of the detection object in the visible light image and the standard image. The multispectral image taken by the unmanned aerial vehicle at the first position is the same as the position of the standard multispectral image. Then, the first environmental data and the first multispectral image are obtained at the first position. Since the environment has a greater influence on the spectrum, it is also necessary to correct the spectral reflectivity of the detection object in the first multispectral image based on the environmental data when the first environmental data and the standard multispectral image are taken, and obtain the first spectral reflectivity curve of the detection object. Finally, based on the first spectral reflectivity curve and the standard spectral reflectivity curve, the detection object is detected. Thereby, not only the detection accuracy of the unmanned aerial vehicle inspection can be improved, but also the detection efficiency can be improved.

[0115] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0116] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.

[0117] Figure 3 The structure diagram of the drone inspection device 300 for the tower components of the power distribution network provided by the embodiment of the present invention is shown. For the convenience of description, only the parts related to the embodiment of the present invention are shown, which are described in detail as follows:

[0118] like Figure 3 As shown, a drone inspection device 300 for tower components of a power distribution network includes:

[0119] A first acquisition module 310 is used to acquire an image set collected in real time, and determine identification information of the detection object on the image set based on the image set; wherein the image set includes a visible light image and a multispectral image of the detection object;

[0120] A determination module 320, configured to determine a first position of the drone based on the position and size of the detected object in the visible light image and the standard image; wherein the standard image is determined from a standard image library based on identification information of the detected object;

[0121] The second acquisition module 330 is used to acquire first environmental data and a first multispectral image; the first multispectral image is taken by the drone at the first position, and the first environmental data is data of the environment in which the drone is located when taking the first multispectral image;

[0122] The processing module 340 is used to correct the spectral reflectance of the detection object in the first multispectral image based on the first environmental data and the environmental data when the standard multispectral image is taken, so as to obtain a first spectral reflectance curve of the detection object;

[0123] The detection module 350 is used to detect the detection object based on the first spectral reflectance curve and the standard spectral reflectance curve.

[0124] In a possible implementation, the first acquisition module 310 is used to input the multispectral image into the multispectral image detection module to obtain the category of the detection object;

[0125] Based on the category of the detected object, the visible light image, and the category boundary detection frame template set, a boundary detection frame of the detected object on the visible light image is determined.

[0126] In a possible implementation, the determination module 320 is configured to filter out a boundary detection frame template corresponding to the detected object from a category boundary detection frame template set based on the category of the detected object;

[0127] Based on the boundary detection frame template corresponding to the detection object, determining the boundary detection frame of the detection object in the standard image containing the detection object;

[0128] The first position of the drone is determined based on the positions of the boundary detection frame on the visible light image and the boundary detection frame in the standard image, and the sizes of the detected objects in the boundary detection frame of the visible light image and the boundary detection frame in the standard image.

[0129] In a possible implementation, the determination module 320 is used to determine the first distance that the drone needs to adjust based on the size of the detection object in the boundary detection frame of the visible light image and the size of the detection object in the boundary detection frame of the standard image; wherein the first distance is the vertical distance between the drone and the center point of the detection object, the size of the detection object in the boundary detection frame on the second visible light image is equal to the size of the detection object in the boundary detection frame in the standard image, and the second visible light image is obtained by the drone after adjusting the first distance;

[0130] Based on the position of the boundary detection frame on the visible light image and the boundary detection frame in the standard image, determine the second distance that the drone needs to adjust; wherein the second distance is the distance from the center point of the camera on the drone projected onto the surface of the detection object to the center point of the detection object, the position of the boundary detection frame on the third visible light image in the third visible light image is the same as the position of the boundary detection frame in the standard image in the standard image, the third visible light image and the standard image have the same size, and the third visible light image is acquired by the drone after adjusting the second distance;

[0131] Based on the first distance and the second distance, a first position of the drone is determined.

[0132] In a possible implementation, the environmental data includes temperature and light, the first environmental data includes a first temperature and a first light, and the standard environmental data includes a standard temperature and a standard light;

[0133] A processing module 340 is used to determine a first change factor based on the first temperature, the standard temperature and the preset temperature spectrum change curve;

[0134] Determining a second variation factor based on the first illumination, the standard illumination, and a preset illumination spectrum variation curve;

[0135] The spectral reflectance of the detection object in the first multispectral image is corrected based on the first variation factor and the second variation factor.

[0136] In a possible implementation, the processing module 340 is used to input the first change factor and the second change factor into a reflectivity correction model to obtain a third change factor; wherein the reflectivity correction model is a neural network model, which is trained based on historical temperature changes and historical light changes;

[0137] The spectral reflectance of the detection object in the first multispectral image is corrected based on the third variation factor.

[0138] In a possible implementation, the standard multispectral image and the standard image are images taken at the same location and containing the detection object, and the standard spectral reflectance curve is the spectral reflectance curve of the detection object in the standard multispectral image;

[0139] A detection module 350 is used to determine that the detected object is abnormal when the difference between the absolute value of the spectral reflectance of the target wavelength in the first spectral reflectance curve and the absolute value of the spectral reflectance in the standard spectral reflectance curve is greater than a preset difference threshold; wherein the target wavelength is any wavelength in the spectral reflectance curve;

[0140] When the sum of the absolute values ​​of the spectral reflectances of all target wavelengths in the first spectral reflectance curve and the absolute values ​​of the spectral reflectances in the standard spectral reflectance curve is greater than a first preset sum threshold, or is less than a second preset sum threshold, it is determined that the detected object is abnormal; wherein the first preset sum threshold is greater than the second preset sum threshold.

[0141] In a possible implementation, the detection module 350 is used to transmit the first multi-spectral image, the standard multi-spectral image, the first spectral reflectance curve, and the standard spectral reflectance curve back to the central server when an abnormality occurs in the detection result.

[0142] Figure 4 The drone provided by an embodiment of the present invention is shown, including a drone inspection device 300 for pole tower components of a distribution network after knowledge distillation and lightweight processing. The drone inspection device 300 for pole tower components of a distribution network is deployed in a drone. The drone inspection device for pole tower components of a distribution network is used to perform inspections based on any method of the first aspect.

[0143] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0144] Those of ordinary skill in the art will appreciate that the templates, units, and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0145] If the module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various drone inspection method embodiments for tower components of the distribution network. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium.

[0146] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for inspecting tower components of a distribution network by using an unmanned aerial vehicle, characterized in that: include: Acquire an image set collected in real time, and determine identification information of the detection object on the image set based on the image set; wherein the image set includes a visible light image and a multispectral image of the detection object; Determining a first position of the drone based on the position and size of the detection object in the visible light image and the standard image; wherein the standard image is determined from a standard image library based on identification information of the detection object; Acquire first environmental data and a first multispectral image; the first multispectral image is taken by the drone at the first position, and the first environmental data is data of the environment in which the drone is located when taking the first multispectral image; Based on the first environmental data and the environmental data when the standard multispectral image is taken, the spectral reflectance of the detection object in the first multispectral image is corrected to obtain a first spectral reflectance curve of the detection object; The detection object is detected based on the first spectral reflectance curve and the standard spectral reflectance curve.

2. The method for inspecting tower components of a power distribution network by using an unmanned aerial vehicle according to claim 1, characterized in that: The determining, based on the image set, identification information of the detection object on the image set comprises: Inputting the multispectral image into a multispectral image detection module to obtain the category of the detected object; Based on the category of the detected object, the visible light image, and a category boundary detection frame template set, a boundary detection frame of the detected object on the visible light image is determined.

3. The method for inspecting tower components of a power distribution network by using an unmanned aerial vehicle according to claim 2, characterized in that: The determining the first position of the drone based on the position and size of the detected object in the visible light image and the standard image includes: Based on the category of the detected object, filter out the boundary detection frame template corresponding to the detected object from the category boundary detection frame template set; Based on the boundary detection frame template corresponding to the detection object, determining the boundary detection frame of the detection object in the standard image containing the detection object; The first position of the drone is determined based on the positions of the boundary detection frame on the visible light image and the boundary detection frame in the standard image, and the sizes of the detected object in the boundary detection frame of the visible light image and the boundary detection frame in the standard image.

4. The method for inspecting tower components of a power distribution network by using an unmanned aerial vehicle according to claim 3, characterized in that: The determining the first position of the drone based on the position of the boundary detection frame on the visible light image and the boundary detection frame in the standard image, and the size of the detected object in the boundary detection frame of the visible light image and the size of the boundary detection frame in the standard image, respectively, comprises: Based on the size of the detected object in the boundary detection frame of the visible light image and the size of the boundary detection frame in the standard image, respectively, determining a first distance that the drone needs to adjust; wherein the first distance is a vertical distance between the drone and the center point of the detected object, the size of the detected object in the boundary detection frame on the second visible light image is equal to the size of the boundary detection frame in the standard image, and the second visible light image is obtained by the drone after adjusting the first distance; Based on the positions of the boundary detection frame on the visible light image and the boundary detection frame in the standard image, determining a second distance that the drone needs to adjust; wherein the second distance is the distance from the center point of the camera on the drone projected onto the surface of the detection object to the center point of the detection object, the position of the boundary detection frame on the third visible light image in the third visible light image is the same as the position of the boundary detection frame in the standard image in the standard image, the third visible light image and the standard image have the same size, and the third visible light image is acquired by the drone after adjusting the second distance; Based on the first distance and the second distance, a first position of the drone is determined.

5. The method for inspecting tower components of a power distribution network by using an unmanned aerial vehicle according to claim 1, characterized in that: The environmental data includes temperature and light, the first environmental data includes a first temperature and a first light, and the standard environmental data includes a standard temperature and a standard light; Based on the first environment data and the standard environment data when the standard multispectral image is captured, the spectral reflectance of the detection object in the first multispectral image is corrected, including: Determining a first variation factor based on the first temperature, the standard temperature, and the preset temperature spectrum variation curve; Determining a second variation factor based on the first illumination, the standard illumination, and a preset illumination spectrum variation curve; The spectral reflectance of the detection object in the first multi-spectral image is corrected based on the first variation factor and the second variation factor.

6. The method for inspecting tower components of a power distribution network by using an unmanned aerial vehicle according to claim 5, characterized in that: The correcting the spectral reflectance of the detected object in the first multispectral image based on the first variation factor and the second variation factor includes: Inputting the first change factor and the second change factor into a reflectivity correction model to obtain a third change factor; wherein the reflectivity correction model is a neural network model, which is trained based on historical temperature changes and historical light changes; The spectral reflectance of the detection object in the first multi-spectral image is corrected based on the third variation factor.

7. The method for inspecting tower components of a power distribution network by using an unmanned aerial vehicle according to claim 1, characterized in that: The standard multispectral image and the standard image are images taken at the same position and containing the detection object, and the standard spectral reflectance curve is the spectral reflectance curve of the detection object in the standard multispectral image; The detecting the detection object based on the first spectral reflectance curve and the standard spectral reflectance curve comprises: When the difference between the absolute value of the spectral reflectance of the target wavelength in the first spectral reflectance curve and the absolute value of the spectral reflectance in the standard spectral reflectance curve is greater than a preset difference threshold, it is determined that the detected object is abnormal; wherein the target wavelength is any wavelength in the spectral reflectance curve; When the sum of the absolute values ​​of the spectral reflectances of all target wavelengths in the first spectral reflectance curve and the absolute values ​​of the spectral reflectances in the standard spectral reflectance curve is greater than a first preset sum threshold, or is less than a second preset sum threshold, it is determined that the detected object is abnormal; wherein the first preset sum threshold is greater than the second preset sum threshold.

8. The method for inspecting tower components of a power distribution network by using an unmanned aerial vehicle according to claim 7, characterized in that: The method further comprises: When an abnormality occurs in the detection result, the first multi-spectral image, the standard multi-spectral image, the first spectral reflectance curve and the standard spectral reflectance curve are transmitted back to the central server.

9. A drone inspection device for tower components of a distribution network, characterized in that: include: A first acquisition module is used to acquire an image set collected in real time, and determine identification information of the detection object on the image set based on the image set; wherein the image set includes a visible light image and a multispectral image of the detection object; A determination module, configured to determine a first position of the drone based on the position and size of the detection object in the visible light image and the standard image; wherein the standard image is determined from a standard image library based on identification information of the detection object; The second acquisition module is used to acquire first environmental data and a first multispectral image; the first multispectral image is taken by the drone at the first position, and the first environmental data is data of the environment in which the drone is located when taking the first multispectral image; A processing module, configured to correct the spectral reflectance of the detection object in the first multispectral image based on the first environmental data and the environmental data when the standard multispectral image is taken, so as to obtain a first spectral reflectance curve of the detection object; The detection module is used to detect the detection object based on the first spectral reflectance curve and the standard spectral reflectance curve.

10. A drone, characterized in that: It includes an unmanned aerial vehicle inspection device for pole tower components of a distribution network after knowledge distillation and lightweight processing, wherein the unmanned aerial vehicle inspection device for pole tower components of a distribution network is deployed in the unmanned aerial vehicle, and the unmanned aerial vehicle inspection device for pole tower components of a distribution network is used to perform inspections based on the method described in any one of claims 1 to 8.

Images