Online insulator spraying method and system based on unmanned aerial vehicle

Abstract

The invention relates to an insulator online spraying method and system based on an unmanned aerial vehicle, and the method comprises the following steps: controlling the unmanned aerial vehicle with a spray gun to be near an insulator to be sprayed, and enabling the unmanned aerial vehicle to collect the video data of the insulator to be sprayed in real time; constructing an insulator target detection model, and inputting the video data of the insulator to be sprayed into the insulator target detection model for insulator detection to obtain an insulator detection result; and an insulator target tracking model is constructed, a target insulator needing to be sprayed is selected from insulator detection results, the unmanned aerial vehicle is controlled to continuously track the target insulator through the insulator target tracking model, the unmanned aerial vehicle is kept at the optimal spraying position, and the insulator spraying task is completed.

Description

Technical Field

[0001] The invention relates to an online spraying method and system for insulators based on unmanned aerial vehicles, and belongs to the technical field of computer vision. Background Art

[0002] The importance of insulator surface coating is mainly reflected in protecting the insulation performance of insulators and extending their service life. As an important component in power transmission lines, insulators bear the role of supporting and fixing conductors, and play a key role in isolating conductors in high-voltage environments. However, external factors such as moisture, pollutants, and dust will erode the surface of the insulator, resulting in a decrease in insulation performance; and the surface paint, as a protective layer, can form a protective layer to prevent these harmful substances from corroding the insulator. Therefore, by spraying the surface paint to protect the insulation performance of the insulator, it can ensure the safe and stable operation of the power transmission line, reduce the failure rate of the insulator, extend its service life, and reduce the operation and maintenance costs.

[0003] However, in the field of power transmission line maintenance, traditional insulator spraying operations usually rely on manual operations. Construction workers need to use insulated boom trucks or climb to transmission towers in high-altitude environments and use manual spraying equipment to coat insulators. This type of operation method has many challenges. The first is the high consumption of human resources. The entire process requires the cooperation of multiple technical workers, and it takes a long time from preparation to actual spraying to quality inspection. Secondly, the work efficiency is low. Since high-altitude operations are complicated and dangerous, workers often need to operate carefully, and the spraying speed is limited. Furthermore, with the rapid development of the national economy, the continuous advancement of production technology and the continuous rise in labor costs, the use of automation technology instead of manpower has become an inevitable trend in inspection management.

[0004] Traditional insulator spraying operations are usually completed manually, which has problems such as high human resource consumption, low work efficiency, and difficulty in ensuring construction quality. Although drone-assisted spraying technology has made certain progress in recent years, the existing spraying methods still have many limitations and cannot fully meet the needs of large-scale applications. In actual operation, although drones can replace some manual spraying work, the existing drone spraying system is still relatively dependent on manual operation, and the operator is required to monitor the spraying path and spraying amount in real time. This semi-automated mode still has high operating difficulty and a long training cycle, resulting in construction efficiency and accuracy that cannot reach the ideal state.

[0005] In addition, the existing drone spraying technology has poor adaptability in complex environments, especially when facing environmental conditions such as high electromagnetic interference or complex terrain, the spraying effect is easily affected, which may lead to inconsistent spraying quality or incomplete coverage. Secondly, the existing drone spraying system generally lacks intelligent analysis and prediction functions, and the implementation effect of the spraying operation still requires manual experience judgment. In actual operations, it is impossible to provide real-time feedback and intelligent adjustment of the surface state of the insulator, resulting in limited operation efficiency and quality. This status quo of relying on on-site operation and monitoring makes the automation level of the drone spraying system low, and the scope of operation is limited, making it difficult to achieve efficient application in large-scale power grid inspections. In general, the current drone spraying technology still mainly relies on basic drone control and sensing equipment, with a small coverage range and poor flexibility, and cannot perform comprehensive and accurate maintenance for the complex environment and diversified needs of power lines. At this stage, the drone spraying system still lacks advanced intelligent means, and fails to make full use of AI technology for operation planning, real-time monitoring and intelligent optimization of spraying effects, which limits its application in large-scale maintenance of power systems. Summary of the invention

[0006] In order to solve the above problems existing in the prior art, the present invention proposes an online spraying method and system for insulators based on drones.

[0007] The technical solution of the present invention is as follows:

[0008] On the one hand, the present invention provides an online spraying method for insulators based on a drone, comprising the following steps:

[0009] Control the drone with the spray gun to the vicinity of the insulator to be sprayed, and the drone collects video data of the insulator to be sprayed in real time;

[0010] An insulator target detection model is constructed, and the video data of the insulator to be sprayed is input into the insulator target detection model to perform insulator detection and obtain the insulator detection result;

[0011] An insulator target tracking model is constructed, and the target insulators that need to be sprayed are selected from the insulator detection results. The insulator target tracking model is used to control the UAV to continuously track the target insulator, so that the UAV remains in the optimal spraying position and completes the insulator spraying task.

[0012] As a preferred embodiment of the present invention, the insulator target detection model is constructed based on the SBB-YOLOv9 model, and the SBB-YOLOv9 model includes a backbone network, a neck network and a head network;

[0013] The backbone network includes several convolutional layers, RepNCSPELAN5-SBB layers and downsampling layers. The input video frame image data is first subjected to feature extraction by the backbone network to generate a preliminary multi-level feature map.

[0014] The neck network includes several convolutional layers, upsampling layers and concatenation layers, which are used to enhance the feature expression of the preliminary multi-level feature map;

[0015] The head network is used to generate feature maps of multiple scales based on the preliminary multi-level feature map of enhanced feature expression;

[0016] The insulator target detection model sets anchor points on feature maps of different scales and draws predicted bounding boxes based on the anchor points. Finally, the insulator target detection model outputs the predicted bounding box coordinates, target category, and target confidence at each scale.

[0017] As a preferred embodiment of the present invention, the insulator target tracking model is constructed based on the Mind-Sort model, which is a DeepSort model in which the original convolutional neural network is replaced with a ResNet34 network, and a LA-Net network is introduced in a specific convolutional layer of the DeepSort model;

[0018] The Mind-Sort model uses MsIoU as the loss function. During the training process of the Mind-Sort model, the loss function is minimized to make the Mind-Sort model predicted bounding box close to its true bounding box. Each predicted bounding box is expressed by the following formula:

[0019] B pr =[x pr ,y pr ,w pr ,h pr ];

[0020] Among them: B pr represents the predicted bounding box; x pr Represents the x-axis coordinate of the center point of the predicted bounding box; y pr represents the y-axis coordinate of the center point of the predicted bounding box; w pr Indicates the width of the predicted bounding box; h pr Represents the height of the predicted bounding box;

[0021] The predicted bounding box is made close to its true bounding box by minimizing the loss function, as shown in the following formula:

[0022] B gt =[x gt ,y gt ,w gt ,h gt ];

[0023] Loss = min γ Loss(B pr ,B gt |γ);

[0024] Among them: B gt represents the true bounding box; x gt Indicates the x-axis coordinate of the center point of the real bounding box; y gt represents the y-axis coordinate of the center point of the real bounding box; w gt Indicates the width of the true bounding box; h gt represents the height of the true bounding box; Loss represents the loss function; γ represents the model parameter.

[0025] As a preferred embodiment of the present invention, when spraying, the drone divides the surface area of ​​the target insulator into different work blocks. When the spraying task of one work block is completed, the drone moves to the next work block to perform the spraying task until the work blocks on one side of the insulator are all sprayed. The drone then moves to the other side of the insulator to perform the same spraying strategy.

[0026] As a preferred embodiment of the present invention, after each spraying operation of a work block is completed, the spraying coverage and spraying uniformity of the work block are evaluated. The specific steps are as follows:

[0027] The drone collects image data of the work piece before spraying and records the initial state of its surface;

[0028] After the spraying is completed, the drone collects image data of the work piece again to record the coating status of its surface;

[0029] Perform spray coverage analysis and spray uniformity analysis based on the image data of the work block before and after spraying, and obtain the spray coverage and spray uniformity evaluation results of the work block;

[0030] The evaluation result is used to determine whether the spraying quality of the current operation block is qualified. If it is unqualified, the spraying is repeated until the spraying quality of the current operation block reaches the standard value.

[0031] As a preferred embodiment of the present invention, the spray coverage analysis is achieved by image difference analysis method, and the specific steps are as follows:

[0032] The grayscale image data or RGB image data of the work block before and after spraying are compared pixel by pixel, and the grayscale value or color value difference ΔI(x,y) of each pixel is calculated, as shown in the following formula:

[0033] ΔI(x,y)=|I before (x,y)-I after (x,y)|;

[0034] Where: I before (x, y) represents the grayscale value or color value of the grayscale image data or RGB image data of the work block before spraying at the pixel position (x, y); after (x, y) represents the grayscale value or color value of the grayscale image data or RGB image data of the work block after spraying at the pixel position (x, y);

[0035] Set a color difference threshold. If the difference ΔI(x,y) of any pixel is greater than the threshold, the pixel is judged to be covered by paint, as shown in the following formula:

[0036]

[0037] Where: D(x,y) is the pixel paint coverage; τ is the color difference threshold;

[0038] The spraying coverage is evaluated by calculating the proportion of pixels with a paint coverage of 1, as shown in the following formula:

[0039]

[0040] Where: W represents the image data width; H represents the image data height;

[0041] Set the spray coverage threshold. If Coverage is less than the preset threshold, it is judged that the spray coverage of the current job block is insufficient.

[0042] As a preferred embodiment of the present invention, the specific steps of the spraying uniformity analysis are:

[0043] Calculate the color standard deviation σ in the RGB image data of the painted work piece isu , as shown in the following formula:

[0044]

[0045] Where: N represents the total number of pixels in the RGB image data; c i represents the color value of the i-th pixel in the RGB image data; μ represents the mean color value of all pixels in the RGB image data;

[0046] Set the standard deviation threshold, if σ isu If it is greater than the preset threshold, it is judged that the spraying uniformity of the current operation block is insufficient.

[0047] On the other hand, the present invention also provides an insulator online spraying system based on a drone, comprising a data acquisition module, an insulator detection module and an insulator tracking module;

[0048] The data acquisition module is used to control a drone with a spray gun to the vicinity of the insulator to be sprayed, and the drone collects video data of the insulator to be sprayed in real time;

[0049] The insulator detection module is used to construct an insulator target detection model, input the video data of the insulator to be sprayed into the insulator target detection model to perform insulator detection, and obtain the insulator detection result;

[0050] The insulator tracking module is used to build an insulator target tracking model, select the target insulator that needs to be sprayed from the insulator detection results, and control the drone to continuously track the target insulator through the insulator target tracking model, so that the drone remains in the optimal spraying position to complete the insulator spraying task.

[0051] On the other hand, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present invention when executing the program.

[0052] In yet another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the present invention.

[0053] The present invention has the following beneficial effects:

[0054] 1. The present invention adopts a lightweight deep neural network model SBB-YOLOv9. The SBB comprehensive branch module can enhance the representation ability of a single convolution, enrich the feature space, improve the detection and reasoning performance of the insulator reasoning model on the edge device, and realize the rapid recognition and positioning of the main target of the insulator; combined with the Mind-Sort algorithm of the original convolutional neural network that replaces DeepSort with ResNet34, it can fully capture information at different levels and optimize the feature extraction ability of the model, so that the detection of the target insulator will not be lost as the drone moves during the spraying process, thereby improving the recognition accuracy and enhancing the stability.

[0055] 2. The present invention combines drone technology and intelligent visual perception technology to analyze the arrangement of insulators in transmission lines, dynamically monitors the collected target insulator images during the spraying process, and uses an insulator spraying quality assessment algorithm to analyze the coating coverage and spraying uniformity, thereby achieving semi-automation of spraying operations assisted by AI, thereby reducing the technical requirements for pilots, improving the efficiency of spraying operations, and providing a more convenient and efficient solution for insulator maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of the method of the present invention;

[0057] Figure 2 This is a structural diagram of the SBB-YOLOv9 model and the Mind-Sort model of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0060] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0061] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0062] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0063] Embodiment 1:

[0064] See also Figure 1 , an insulator online spraying method based on drone, comprising the following steps:

[0065] Control the drone with the spray gun to the vicinity of the insulator to be sprayed, and the drone collects video data of the insulator to be sprayed in real time;

[0066] An insulator target detection model is constructed, and the video data of the insulator to be sprayed is input into the insulator target detection model to perform insulator detection and obtain the insulator detection result;

[0067] An insulator target tracking model is constructed, and the target insulators to be sprayed are selected from the insulator detection results. The UAV is controlled by the insulator target tracking model to continuously track the target insulator. According to the changes in the UAV's video frame, the relative position of the insulator in the camera's field of view is calculated (such as the deviation between the image center and the target label box (the target label box can provide the center coordinates of the insulator for flight path planning)), and the proportional-integral controller is used to adjust the position of the UAV in real time to keep the UAV in the optimal spraying position (for example, facing the front of the target and within the appropriate range of the nozzle distance). According to the real-time data of target tracking, the lateral, longitudinal position and yaw angle of the UAV are adjusted to ensure that the UAV maintains the optimal spraying distance and angle of the insulator and completes the insulator spraying task.

[0068] As a preferred implementation of this embodiment, see Figure 2 , the insulator target detection model is built based on the SBB-YOLOv9 model, and the SBB-YOLOv9 model includes a backbone network, a neck network and a head network;

[0069] The backbone network includes several convolutional layers, RepNCSPELAN5-SBB layers and downsampling layers. The input video frame image data is first subjected to feature extraction by the backbone network to generate a preliminary multi-level feature map. The RepNCSPELAN5-SBB layer is embedded with a synthetic branch block (SBB) based on the fusion of the CSPNet Block module of YOLOv5, the Rep module of YOLOv6 and the ELAN module of YOLOv7.

[0070] The neck network includes several convolutional layers, upsampling layers and concatenation layers, which are used to enhance the feature expression of the preliminary multi-level feature map;

[0071] The head network is used to generate feature maps of multiple scales based on the preliminary multi-level feature map of enhanced feature expression;

[0072] The insulator target detection model sets anchors on feature maps of different scales and draws predicted bounding boxes based on the anchors. Finally, the insulator target detection model outputs the predicted bounding box coordinates, target category, and target confidence at each scale.

[0073] The SBB structure can enhance the representation capability of a single convolution, enrich the feature space, improve the feature extraction capability of the model, and enhance the model performance, while basically not increasing any reasoning time cost;

[0074] The SBB structure enhances the representation capability of a single convolution and enriches the feature space by combining diverse branches of different scales and complexities, including convolution sequences, multi-scale convolutions, and average pooling.

[0075] After training, the SBB structure can be equivalently converted to a single convolutional layer for deployment. Unlike the ConvNet architecture, the SBB structure complicates the microstructure during training while maintaining the macrostructure, so it can be used as a plug-and-play replacement for regular convolutional layers in any architecture. In this way, the model can be trained to a higher performance level and then converted back to the original reasoning time structure for reasoning;

[0076] The SBB structure combines branches of different scales and complexities, such as convolution kernels of different sizes and average pooling, to increase the feature expression capability of a single convolution. During the training phase, the SBB structure adopts a complex branch structure, while during the inference phase, these branches can be equivalently converted into a single convolution layer to maintain efficient inference. The SBB structure allows it to be inserted into an existing network as a substitute for a conventional convolution layer without changing the overall network architecture.

[0077] As a preferred implementation of this embodiment, see Figure 2 The insulator target tracking model is constructed based on the Mind-Sort model. The Mind-Sort model is a DeepSort model that replaces the original convolutional neural network with a ResNet34 network (ResNet34 is a deep learning model whose unique residual structure enables it to have a more powerful feature extraction capability, which can help the network better understand the appearance information of the target), and introduces the LA-Net network in a specific convolutional layer of the DeepSort model (the LA-Net network is a simple channel attention mechanism that can greatly optimize the feature extraction capability of the model while adding a small number of parameters). In this embodiment, the LA-Net network is introduced in the third, ninth and thirteenth convolutional layers, respectively, and the features at different levels are connected through the residual module;

[0078] This design can fully capture information at different levels, so that the target insulators will not be lost as the drone moves during the spraying process, improving the recognition accuracy while enhancing stability;

[0079] The Mind-Sort model uses MsIoU as the loss function. MsIoU (Multi-scale Intersection over Union) is an indicator for evaluating the performance of image segmentation models. It is an improvement on the conventional IoU (Intersection over Union), especially used in multi-scale object image segmentation tasks. MsIoU can more accurately reflect the difference between the predicted box and the true box by directly calculating the key point distance between the predicted box and the true box, thereby improving the average accuracy of the model. During the training process of the Mind-Sort model, the Mind-Sort model predicts the bounding box close to its true bounding box by minimizing the loss function. Each predicted bounding box is represented by the following formula:

[0080] B pr =[x pr ,y pr ,w pr ,h pr ];

[0081] Among them: B pr represents the predicted bounding box; x pr Represents the x-axis coordinate of the center point of the predicted bounding box; y pr represents the y-axis coordinate of the center point of the predicted bounding box; w pr Indicates the width of the predicted bounding box; h pr Represents the height of the predicted bounding box;

[0082] The predicted bounding box is made close to its true bounding box by minimizing the loss function, as shown in the following formula:

[0083] B gt =[x gt ,y gt ,w gt ,h gt ];

[0084] Loss = min γ Loss(B pr ,B gt |γ);

[0085] Among them: B gt represents the true bounding box; x gt Indicates the x-axis coordinate of the center point of the real bounding box; y gt represents the y-axis coordinate of the center point of the real bounding box; w gt Indicates the width of the true bounding box; h gt represents the height of the true bounding box; Loss represents the loss function; γ represents the model parameter.

[0086] As a preferred implementation mode of this embodiment, the drone divides the surface area of ​​the target insulator into different work blocks when spraying. When the spraying task of one work block is completed, the drone moves to the next work block to perform the spraying task until the work blocks on one side of the insulator are all sprayed. The drone then moves to the other side of the insulator to perform the same spraying strategy.

[0087] The arrangement of insulators in transmission lines is mainly divided into two types: horizontal and vertical. For horizontally arranged insulators, set the drone to fly along the center line of the insulator and maintain a flight height parallel to the insulator to ensure uniform spray coverage. When above each insulator, the spray angle and nozzle height should be adjusted to ensure that the paint can cover the sides and top of the insulator. At the same time, set the spraying time and dynamically adjust it according to the area of ​​the insulator to ensure the uniformity and coverage of the paint. After finishing the spraying operation on one side of the insulator, set the drone to fly to the other side for spraying, covering the front and back sides of the target insulator, so as to take into account all parts of the insulator.

[0088] Similarly, when spraying insulators arranged vertically, set the drone to fly along the vertical center line of the insulator, choose to spray layer by layer from top to bottom, and maintain a stable flight altitude. After spraying on one side, control the drone to fly to the other side to spray, so as to cover the left and right sides of each insulator.

[0089] As a preferred implementation of this embodiment, after each spraying operation of a work block is completed, the spraying coverage and spraying uniformity of the work block are evaluated. The specific steps are as follows:

[0090] The drone collects image data of the work piece before spraying and records the initial state of its surface;

[0091] After the spraying is completed, the drone collects image data of the work piece again to record the coating status of its surface;

[0092] Perform spray coverage analysis and spray uniformity analysis based on the image data of the work block before and after spraying, and obtain the spray coverage and spray uniformity evaluation results of the work block;

[0093] The evaluation result is used to determine whether the spraying quality of the current operation block is qualified. If it is unqualified, the spraying is repeated until the spraying quality of the current operation block reaches the standard value.

[0094] As a preferred implementation of this embodiment, the spray coverage analysis is implemented by image difference analysis method, and the specific steps are as follows:

[0095] The grayscale image data or RGB image data of the work block before and after spraying are compared pixel by pixel, and the grayscale value or color value difference ΔI(x,y) of each pixel is calculated, as shown in the following formula:

[0096] ΔI(x,y)=|I before (x,y)-I after (x,y)|;

[0097] Where: I before (x, y) represents the grayscale value or color value of the grayscale image data or RGB image data of the work block before spraying at the pixel position (x, y); after (x, y) represents the grayscale value or color value of the grayscale image data or RGB image data of the work block after spraying at the pixel position (x, y);

[0098] Set a color difference threshold. If the difference ΔI(x,y) of any pixel is greater than the threshold, the pixel is judged to be covered by paint, as shown in the following formula:

[0099]

[0100] Where: D(x,y) is the pixel paint coverage; τ is the color difference threshold; this step will generate a binary difference image, indicating the coverage of the spray paint;

[0101] The spray coverage is evaluated by calculating the proportion of pixels with a value of 1 in the binary difference image, as shown in the following formula:

[0102]

[0103] Where: W represents the image data width; H represents the image data height;

[0104] Set the spray coverage threshold. If Coverage is less than the preset threshold, it is judged that the spray coverage of the current job block is insufficient.

[0105] As a preferred implementation of this embodiment, the specific steps of the spraying uniformity analysis are:

[0106] Calculate the color standard deviation σ in the RGB image data of the painted work piece isu , as shown in the following formula:

[0107]

[0108] Where: N represents the total number of pixels in the RGB image data; c i represents the color value of the i-th pixel in the RGB image data; μ represents the mean color value of all pixels in the RGB image data;

[0109] Set the standard deviation threshold, if σ isu If it is greater than the preset threshold, it is judged that the spraying uniformity of the current operation block is insufficient;

[0110] In this embodiment, the color histogram of the image after spraying can be calculated to analyze whether the color distribution of the coating is uniform. A uniform coating should have a relatively smooth histogram curve without obvious peaks. Alternatively, the texture features of the coating can be extracted using the Haralick texture feature algorithm. If the texture of the coating is too complex or uneven, it may indicate uneven spraying.

[0111] Embodiment 2:

[0112] An insulator online spraying system based on a drone, comprising a data acquisition module, an insulator detection module and an insulator tracking module;

[0113] The data acquisition module is used to control a drone with a spray gun to the vicinity of the insulator to be sprayed, and the drone collects video data of the insulator to be sprayed in real time;

[0114] The insulator detection module is used to construct an insulator target detection model, input the video data of the insulator to be sprayed into the insulator target detection model to perform insulator detection, and obtain the insulator detection result;

[0115] The insulator tracking module is used to build an insulator target tracking model, select the target insulator that needs to be sprayed from the insulator detection results, and control the drone to continuously track the target insulator through the insulator target tracking model, so that the drone remains in the optimal spraying position to complete the insulator spraying task.

[0116] The system is used to implement the method in Example 1, which will not be described in detail here.

[0117] Embodiment three:

[0118] This embodiment provides an electronic device, including a drone and a matching remote controller with a screen.

[0119] In this embodiment, the drone has a built-in front-end AI module for implementing the method described in Embodiment 1. The front-end AI module can identify the spray target by analyzing the video stream and generate corresponding flight control instructions to command the drone to complete precise operations.

[0120] The remote control not only allows the operator to monitor the flight and spraying of the drone in real time, but can also be used to send key commands, such as starting spraying with one click;

[0121] The spray gun includes a flexible gimbal and a gun body module, which can realize three-dimensional, comprehensive and dead-angle spraying operations in the air. It has adjustable angles of ±30° up and down and ±90° left and right, and through a combination of multiple degrees of freedom, it can flexibly reach the operating target and adapt to complex and narrow spaces without being restricted by flight paths and space; at the same time, it meets the maximum liquid load of 2L, is suitable for 3kg-class drones, is easy to transport and deploy, and is suitable for large / small flight spaces, meeting the needs of precise spraying operations under light-load platforms.

[0122] The front-end AI module is equipped with a 384-core NVIDIA Volta with 48 Tensor cores. TM GPU, 6-core NVIDIA Carmel v8.2 64-bit CPU 6MB L2+4MB L3, three power levels of 10W | 15W | 20W, its GPU performance is very good and the power consumption is low. Only 70mm × 45mm, the powerful performance of NVIDIA Xavier SoC can be packaged into a module the size of Jetson Nano. This compact module combines excellent performance and power consumption advantages with a rich set of IOs (from high-speed CSI and PCIe to low-speed I2C and GPIO). Bring new capabilities to all embedded AI and edge systems with a small form factor, numerous sensor interfaces and excellent performance. Up to 21TOPS of accelerated computing power can run modern neural networks in parallel and process data from multiple high-resolution sensors.

[0123] Cloud-native support for front-end AI modules enables developers to build and deploy high-quality, software-defined functions on embedded and edge devices. NVIDIA NGC's pre-trained AI models and TAO toolkit accelerate the implementation of trained and optimized AI networks and deploy them to modules in a containerized manner, enabling flexible and seamless updates. In addition, the low power consumption and stability make the front-end AI module suitable for long-term operation at the transmission line inspection site, providing reliable support for safety monitoring and real-time decision-making.

[0124] First, the front-end AI module detects the position of the insulator in real time through the SBB-YOLOv9 target detection model, identifies each insulator in the video stream, and provides basic data for subsequent target tracking. After selecting the target to be sprayed, the front-end AI module uses the Mind-Sort target tracking algorithm to continuously track the insulator to ensure that the drone can accurately maintain the optimal spraying position. At the same time, the front-end AI module dynamically plans the flight path of the drone based on the detected insulator position, and adjusts the lateral, longitudinal position and yaw angle of the drone in real time through the proportional integral controller (PID) to maintain the appropriate spraying distance and angle.

[0125] During the spraying process, the front-end AI module is responsible for dividing the insulator surface into different work blocks and directing the drone to complete the spraying work block by block. At the same time, according to the coating coverage and uniformity evaluation results, the front-end AI module can also dynamically adjust the drone's spraying parameters and path planning to ensure that the spraying effect of each work block meets the standard. This series of functions enables the front-end AI module to fully assist the drone in completing the insulator spraying task and realize intelligent and high-precision operation.

[0126] Efficient and precise spraying operations can only be achieved on the basis of balancing weight and flight performance, ensuring device stability and operational capabilities, and providing reliable power supply and communication integration. To address the pain points of these aerial spraying scenarios, a spray gun combined with the core Pliabot flexible technology is used. This spray gun can be adapted to light-load UAV platforms such as DJI M300 / M350, and can achieve flexible and safe operations in narrow spaces in low-altitude scenarios. It uses unique flexible joints to achieve three-dimensional, multi-faceted, and dead-angle spraying. At the same time, it provides a safe, stable, convenient, and efficient multi-angle, refined, and lightweight aerial spraying solution in the hovering / flying state, which solves the problem of aerial spraying operation and maintenance and greatly reduces the safety risks of manual operations.

[0127] Embodiment 4:

[0128] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any embodiment of the present invention is implemented.

[0129] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0130] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0132] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0133] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An online spraying method for insulators based on drones, characterized in that: The following steps are involved: Control the drone with the spray gun to the vicinity of the insulator to be sprayed, and the drone collects video data of the insulator to be sprayed in real time; An insulator target detection model is constructed, and the video data of the insulator to be sprayed is input into the insulator target detection model to perform insulator detection and obtain the insulator detection result; An insulator target tracking model is constructed, and the target insulators that need to be sprayed are selected from the insulator detection results. The insulator target tracking model is used to control the UAV to continuously track the target insulator, so that the UAV remains in the optimal spraying position and completes the insulator spraying task.

2. The method for online spraying of insulators based on drones according to claim 1 is characterized in that: The insulator target detection model is built based on the SBB-YOLOv9 model, and the SBB-YOLOv9 model includes a backbone network, a neck network and a head network; The backbone network includes several convolutional layers, RepNCSPELAN5-SBB layers and downsampling layers. The input video frame image data is first subjected to feature extraction by the backbone network to generate a preliminary multi-level feature map. The neck network includes several convolutional layers, upsampling layers and concatenation layers, which are used to enhance the feature expression of the preliminary multi-level feature map; The head network is used to generate feature maps of multiple scales based on the preliminary multi-level feature map of enhanced feature expression; The insulator target detection model sets anchor points on feature maps of different scales and draws predicted bounding boxes based on the anchor points. Finally, the insulator target detection model outputs the predicted bounding box coordinates, target category, and target confidence at each scale.

3. The method for online spraying of insulators based on drones according to claim 1 is characterized in that: The insulator target tracking model is constructed based on the Mind-Sort model, which is a DeepSort model that replaces the original convolutional neural network with a ResNet34 network and introduces a LA-Net network in a specific convolutional layer of the DeepSort model; The Mind-Sort model uses MsIoU as the loss function. During the training process of the Mind-Sort model, the loss function is minimized to make the Mind-Sort model predicted bounding box close to its true bounding box. Each predicted bounding box is expressed by the following formula: B pr =[x pr ,y pr ,w pr ,h pr ]; Among them: B pr represents the predicted bounding box; x pr Represents the x-axis coordinate of the center point of the predicted bounding box; y pr represents the y-axis coordinate of the center point of the predicted bounding box; w pr Indicates the width of the predicted bounding box; h pr Represents the height of the predicted bounding box; The predicted bounding box is made close to its true bounding box by minimizing the loss function, as shown in the following formula: B gt =[x gt ,y gt ,w gt ,h gt ]; Loss=min γ Loss(B pr ,B gt |γ); Among them: B gt represents the true bounding box; x gt Indicates the x-axis coordinate of the center point of the real bounding box; y gt represents the y-axis coordinate of the center point of the real bounding box; w gt Indicates the width of the true bounding box; h gt represents the height of the true bounding box; Loss represents the loss function; γ represents the model parameter.

4. The method for online spraying of insulators based on drones according to claim 1 is characterized in that: When spraying, the drone divides the surface area of ​​the target insulator into different work blocks. When the spraying task of one work block is completed, the drone moves to the next work block to perform the spraying task until the work blocks on one side of the insulator are all sprayed. The drone then moves to the other side of the insulator to perform the same spraying strategy.

5. The method for online spraying of insulators based on drones according to claim 4 is characterized in that: After each spraying operation of a work block is completed, the spraying coverage and spraying uniformity of the work block are evaluated. The specific steps are as follows: The drone collects image data of the work piece before spraying and records the initial state of its surface; After the spraying is completed, the drone collects image data of the work piece again to record the coating status of its surface; Perform spray coverage analysis and spray uniformity analysis based on the image data of the work block before and after spraying, and obtain the spray coverage and spray uniformity evaluation results of the work block; The evaluation result is used to determine whether the spraying quality of the current operation block is qualified. If it is unqualified, the spraying is repeated until the spraying quality of the current operation block reaches the standard value.

6. The method for online spraying of insulators based on drones according to claim 5 is characterized in that: The spray coverage analysis is achieved by image difference analysis, and the specific steps are as follows: The grayscale image data or RGB image data of the work block before and after spraying are compared pixel by pixel, and the grayscale value or color value difference ΔI(x,y) of each pixel is calculated, as shown in the following formula: ΔI(x,y)=|I before (x,y)-I after (x,y)|; Where: I before (x, y) represents the grayscale value or color value of the grayscale image data or RGB image data of the work block before spraying at the pixel position (x, y); after (x, y) represents the grayscale value or color value of the grayscale image data or RGB image data of the work block after spraying at the pixel position (x, y); Set a color difference threshold. If the difference ΔI(x,y) of any pixel is greater than the threshold, the pixel is judged to be covered by paint, as shown in the following formula: Where: D(x,y) is the pixel paint coverage; τ is the color difference threshold; The spraying coverage is evaluated by calculating the proportion of pixels with a paint coverage of 1, as shown in the following formula: Where: W represents the image data width; H represents the image data height; Set the spray coverage threshold. If Coverage is less than the preset threshold, it is judged that the spray coverage of the current job block is insufficient.

7. The method for online spraying of insulators based on drones according to claim 5, characterized in that: The specific steps of the spray uniformity analysis are: Calculate the color standard deviation σ in the RGB image data of the painted work piece isu , as shown in the following formula: Where: N represents the total number of pixels in the RGB image data; c i represents the color value of the i-th pixel in the RGB image data; μ represents the mean color value of all pixels in the RGB image data; Set the standard deviation threshold, if σ isu If it is greater than the preset threshold, it is judged that the spraying uniformity of the current operation block is insufficient.

8. An insulator online spraying system based on drone, characterized in that: It includes a data acquisition module, an insulator detection module and an insulator tracking module; The data acquisition module is used to control a drone with a spray gun to the vicinity of the insulator to be sprayed, and the drone collects video data of the insulator to be sprayed in real time; The insulator detection module is used to construct an insulator target detection model, input the video data of the insulator to be sprayed into the insulator target detection model to perform insulator detection, and obtain the insulator detection result; The insulator tracking module is used to build an insulator target tracking model, select the target insulator that needs to be sprayed from the insulator detection results, and control the drone to continuously track the target insulator through the insulator target tracking model, so that the drone remains in the optimal spraying position to complete the insulator spraying task.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images