Unmanned aerial vehicle inspection whole process tree barrier real-time analysis and judgment method, device and medium

Abstract

This invention discloses a method, equipment, and medium for real-time analysis and judgment of tree obstacles during the entire process of drone inspection. The method includes the following steps: Step S1: Extract several candidate bounding boxes from images captured by the drone; Step S2: Scale all candidate bounding boxes to a fixed size; Step S3: Extract features from each candidate bounding box using a DCNN to obtain a feature vector of fixed length; Step S4: Input the feature vector into an SVM for binary classification to distinguish between power lines and tree obstacles and mark the location of tree obstacles in the image; Step S5: Perform regression to obtain the coordinate information of the corresponding location of the tree obstacle. This invention enables the identification of tree obstacles during drone channel inspection, allowing the drone to effectively avoid tree obstacles causing overhead cables while maximizing the inspection of cables within tree obstacles.

Description

Technical Field

[0001] This invention belongs to the field of power line inspection technology, specifically relating to drone inspection technology. Background Technology

[0002] In recent years, my country's power grid construction has continued to expand, and traditional manual inspection methods can no longer meet the extensive needs of modern power supply systems. To promote intelligent operation and maintenance technology reform and improve the efficiency of distribution network operation and maintenance, drones are increasingly being used in distribution network line inspection operations. The application of drones has enabled high efficiency and intelligence in distribution network lines, supplementing the absence of manual inspections.

[0003] During existing drone inspections, they are often interfered with by many obstacles, which not only pose a danger to the drones but also interfere with the image capture of the cables. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a method for real-time analysis and judgment of tree obstacles during the entire process of UAV inspection, which can identify tree obstacles that are harmful to the overhead power distribution lines during the inspection process.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A method for real-time analysis and judgment of tree obstacles during the entire process of drone inspection includes the following steps:

[0007] Step S1: Extract several region candidate boxes from the images captured by the drone;

[0008] Step S2: Scale all region candidate boxes to a fixed size;

[0009] Step S3: Use DCNN to extract features from each candidate box to obtain a fixed-length feature vector;

[0010] Step S4: Feed the feature vector into SVM for binary classification to distinguish between power lines and tree obstacles and mark the location of tree obstacles in the image;

[0011] Step S5: Perform regression to obtain the coordinate information of the corresponding location of the tree barrier.

[0012] Preferably, in step S1, the Selective Search algorithm is used to extract more than 2,000 uniformly distributed image frames from the image.

[0013] Preferably, in step S2, all candidate regions are scaled to 227*227.

[0014] Preferably, the tree obstacle candidate box image obtained in step S2 is used as a label for neural network training to train a new model for tree obstacle analysis of the next UAV inspection channel, and the tree obstacle corresponding position coordinate information obtained in step S5 is stored as a label for tree obstacle analysis of the next UAV inspection channel.

[0015] Preferably, the convolutional network used for feature extraction in step S3 has 5 convolutional layers and 2 fully connected layers. Its input is a fixed-size RGB image, and its output is a 4096-dimensional feature vector.

[0016] Preferably, the images captured by the drone are transmitted back to the 5G mobile edge computing node in real time for analysis and judgment.

[0017] Preferably, a semantic segmentation neural network based on deep learning technology is used to segment the wire portion in the image captured by the drone, and the direction and center point position of the segmented portion are fitted. Based on the direction and position deviation, the drone is controlled.

[0018] The present invention also provides a computer device, including at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to execute the aforementioned method for real-time analysis and judgment of tree obstacles throughout the entire process of UAV inspection.

[0019] The present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the aforementioned method for real-time analysis and judgment of tree obstacles throughout the entire process of UAV inspection.

[0020] The technical solution adopted in this invention transmits images in real time to a 5G mobile edge computing node during drone operations. AI algorithms then determine the presence of tree obstacles in real time, enabling the drone to identify these obstacles during channel inspections. This allows the drone to effectively avoid tree-related obstacles such as overhead cables while simultaneously inspecting cables within those obstacles as much as possible. This improves operational speed and ensures operational safety, helping companies optimize their personnel structure from a "622" work model to a "322" model, reducing inspection personnel by 30% and lowering personnel training costs. Furthermore, the improved flight efficiency helps extend the drone's lifespan, saving 300 sets of drone batteries annually, thus reducing operation and maintenance costs and equipment investment.

[0021] The specific technical solution of the present invention and its beneficial effects will be described in detail in the following specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0022] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0023] Figure 1 This is a flowchart of the method for real-time analysis and judgment of tree obstacles during the entire process of drone inspection, as described in this invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] With the development of AI intelligent algorithms and 5G technology, applying 5G technology to power line inspection by drones will improve the overall efficiency of the operation, thereby unlocking more difficulties in tree obstacle identification, and realizing assisted flight control for drone inspection and intelligent judgment of line passages.

[0026] This invention is based on existing drone-based cable inspection technology. It utilizes a drone to inspect cables, with a camera on the bottom of the drone capturing optical images. First, a semantic segmentation neural network based on deep learning is used to segment the overhead power line portion of the image. For each segmented portion, the direction and center point position are fitted. Based on the direction and position deviations, the drone is controlled. The image of the segmented overhead power line portion is then extracted. Due to the significant color difference between the overhead power line and tree branches, SVM binary classification can be performed only on the straight sections of the overhead power line to distinguish between power lines and tree obstacles, and the location of the tree obstacles is marked in the image.

[0027] Example 1

[0028] like Figure 1 As shown, a method for real-time analysis and judgment of tree obstacles during the entire process of drone inspection is described. The overall process of this embodiment includes the following steps:

[0029] Step S1: The drone collects images in real time during the operation and uses 5G network and edge computing to transmit the images to the local end. After being read locally, the Selective Search algorithm is used to extract about 2,000 region candidate boxes from the original image. These candidate boxes may contain tree obstacles. These image boxes are evenly distributed.

[0030] Step S2: Scale all candidate boxes to a fixed size to achieve a preliminary segmentation effect, specifically according to 227*227.

[0031] Step S3: Further process the segmented image by using DCNN to extract features from each candidate box, resulting in a fixed-length feature vector.

[0032] To more accurately identify targets of different sizes, this invention adopts a residual dual-scale detector algorithm model. The backbone network of the model front end adopts the basic ResNet101 network model, and a pyramid architecture is used to extract features of different size dimensions.

[0033] Step S4: Feed the feature vector into an SVM (Support Vector Machine) for classification to obtain category information. Since the pixel information of the overhead power line section and the tree barrier section in the image is different, the overhead power line section can be classified separately using SVM to distinguish between the tree barrier section and the overhead power line section. SVM has good generalization ability and can reduce the amount of computation and speed up the system's operation compared to a complete classification neural network. It is a suitable choice for binary classification problems.

[0034] Step S5: After the line tower detection is completed, the tree obstacles that extend beyond the line are marked and sent to the fully connected network for regression to obtain the corresponding location coordinate information.

[0035] Step S6: Save the corresponding location coordinates as a tag for the next drone inspection of the tree obstacle analysis.

[0036] The challenge of this invention lies in identifying tree obstacles that pose a threat to overhead distribution lines, based on segmentation. Since overhead distribution lines occupy relatively few pixels in actual images, extracting information about them is quite difficult. Therefore, a semantic segmentation model is first used to extract a rectangular region composed of multiple overhead distribution lines. This extracted rectangular region is then further segmented to pinpoint the location of the wires within it.

[0037] Because the pixel information of overhead power distribution lines and tree obstacles in images is different, and tree obstacles are often located above overhead power distribution lines, obstructing them, the pixel information of the overhead power distribution line area is not as singular as before. Therefore, this invention uses a linear support vector machine to calculate the feature vectors of all candidate regions for each image to be detected, and then feeds them into the support vector machine for binary classification. If there is occlusion, the result will be two categories: one is overhead power distribution lines, and the other is tree obstacles, thus realizing the identification of tree obstacles.

[0038] This invention analyzes tree obstacles in drone inspection routes. The algorithm model used is continuously updated and improved based on feedback from drone flight data. Specifically, the candidate tree obstacle images obtained in step S2 are used as labels for neural network training, training a new model for subsequent drone inspection route tree obstacle analysis.

[0039] In step S3, the convolutional network used to extract features has 5 convolutional layers and 2 fully connected layers. Its input is a fixed-size RGB image, and its output is a 4096-dimensional feature vector.

[0040] This invention discloses a real-time tree obstacle analysis and judgment method for the entire process of drone inspection. Deployed on a central node of 5G mobile edge computing, the method transmits captured images back to the 5G mobile edge computing node in real time for analysis and judgment during drone operations. Utilizing 5G as the communication method replaces the current 4G method for transmitting video and control commands, reducing latency in transmitting high-definition images from the drone and improving image clarity. Simultaneously, it employs the Huawei Atlas200 module to provide GPU computing power. Through AI algorithms and 5G technology, intelligent analysis of tree obstacle defects during drone inspection is achieved. 5G networks and edge computing ensure the transmission requirements of drone industry applications. Using MEC-based edge routing and edge computing capabilities, it provides ultra-low latency guarantee services, routing services, edge computing services, and security services for drone applications.

[0041] After analyzing and judging tree obstacles in real time during the entire drone inspection process using the above-mentioned method, if a tree obstacle is determined to exist, the inspection drone will adjust its flight path when it reaches that point, passing over the obstacle. The inspection drone that has passed the obstacle will then return to its original flight path and continue inspection until the end of the flight path.

[0042] This invention enables line inspection drones to effectively avoid tree obstacles that obstruct overhead cables, while simultaneously completing line inspections within tree obstacles as much as possible.

[0043] Example 2

[0044] A computer device, characterized in that it includes at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to execute a real-time tree obstacle analysis and judgment method for the entire process of UAV inspection as described in Embodiment 1.

[0045] Example 3

[0046] A computer-readable storage medium storing computer-executable instructions, wherein when a processor executes the computer-executable instructions, a method for real-time analysis and judgment of tree obstacles throughout the entire process of UAV inspection is implemented as described in Embodiment 1.

[0047] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art should understand that the present invention includes, but is not limited to, the content described in the above specific embodiments. Any modifications that do not depart from the functional and structural principles of the present invention will be included within the scope of the claims.

Claims

1. A method for real-time analysis and judgment of tree obstacles during the entire process of drone inspection, characterized in that, Includes the following steps: Step S1: Extract several region candidate boxes from the images captured by the drone; Step S2: Scale all region candidate boxes to a fixed size; Step S3: Use DCNN to extract features from each candidate box to obtain a fixed-length feature vector; Step S4: Feed the feature vector into SVM for binary classification to distinguish between power lines and tree obstacles and mark the location of tree obstacles in the image; Step S5: Perform regression to obtain the coordinate information of the corresponding location of the tree barrier; In step S1, the Selective Search algorithm is used to extract more than 2000 uniformly distributed image boxes from the image. In step S2, all candidate boxes are scaled to 227*227. The tree obstacle candidate box images obtained in step S2 are used as labels for neural network training to train a new model for tree obstacle analysis in the next drone inspection channel. The tree obstacle location coordinate information obtained in step S5 is stored as labels for tree obstacle analysis in the next drone inspection channel. In step S3, the convolutional network used for feature extraction has 5 convolutional layers and 2 fully connected layers. Its input is a fixed-size RGB image, and its output is a 4096-dimensional feature vector. The images captured by the drone are transmitted back to the 5G mobile edge computing node in real time for real-time analysis and judgment. The semantic segmentation neural network based on deep learning technology is used to segment the wire parts in the images captured by the drone. The direction and center point position of the segmented parts are fitted. Based on the direction and position deviation, the drone is controlled.

2. A computer device, characterized in that, It includes at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to execute the real-time tree obstacle analysis and judgment method for the entire process of UAV inspection as described in claim 1.

3. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, it implements the method for real-time analysis and judgment of tree obstacles throughout the entire process of UAV inspection as described in claim 1.

Images