An overhead ground wire broken strand type unmanned aerial vehicle inspection defect sample generation method

Abstract

The application discloses a kind of overhead ground wire broken strand class unmanned aerial vehicle inspection defect sample generation method, randomly selects a panorama from high dynamic range panorama library as light source, generates defect ground wire three-dimensional rendering model through defect ground wire generation module, randomly selects metal material from ground wire PBR material library and then generates new seamless map after multi-scale fusion, maps material to defect ground wire three-dimensional rendering model by UV mapping method, generates image using ray tracing progressive rendering mode, converts defect part bounding box three-dimensional space coordinates into camera two-dimensional image coordinates by defect automatic labeling module, generates defect labeling frame;The method can provide a large amount of data support for the development of defect detection intelligent algorithm.

Description

Technical Field

[0001] This invention belongs to the fields of automated power line inspection and image processing, specifically relating to a method for generating defect samples of broken strands in overhead conductors and ground wires using unmanned aerial vehicles (UAVs). Background Technology

[0002] Overhead conductor and ground wire breaks have always been a significant threat to the safe and stable operation of power transmission networks, and such incidents occur frequently. Before a conductor or ground wire breaks, there are usually defects of varying degrees, such as broken strands, loose strands, and corrosion. Broken strands and loose strands are often more common in conductors and ground wires, and the causes vary.

[0003] During routine inspections, frontline teams, limited by perspective and line-of-sight, can only occasionally detect broken or scattered strands in conductors using traditional methods. Often, these are critical issues, resulting in significant oversights, particularly minor problems. Therefore, conducting close-range, detailed inspections of conductors using drones, and intelligently identifying defects, is of great practical significance in preventing conductor breakage accidents.

[0004] The structural characteristics of conductors and ground wires after breakage or disintegration are complex and diverse, making it easy for existing methods to produce false positives and false negatives, and difficult to accurately identify the defect status of overhead conductors and ground wires. Developing an intelligent defect recognition algorithm for conductor and ground wire images requires a large number of conductor and ground wire defect samples. Summary of the Invention

[0005] To address the problem of limited and monotonous sample sizes of broken conductor / ground wire defects during current UAV inspections, this invention provides a method for generating UAV inspection defect samples of broken conductor / ground wires.

[0006] The technical solution adopted by this invention to solve its technical problem is: a method for generating defect samples of overhead conductor ground wire broken strands by UAV inspection. The sample generation system consists of a high dynamic range panoramic image library, a defect conductor ground wire generation module, a conductor ground wire PBR material library, an automatic defect annotation module, and a ray tracing-based 3D rendering model of the defect conductor ground wire. The sample generation system runs on a 3D rendering server and includes the following steps when generating defect samples of overhead conductor ground wire broken strands by UAV inspection: The sample generation process randomly selects a panoramic image from the high dynamic range panoramic image library as the light source, generates a defect conductor ground wire 3D rendering model through the defect conductor ground wire generation module, randomly selects a metal material from the conductor ground wire PBR material library and generates a new seamless texture through multi-scale fusion, maps the material onto the defect conductor ground wire 3D rendering model through UV mapping, generates an image using ray tracing progressive rendering, and converts the 3D spatial coordinates of the defect location bounding box to the 2D image coordinates of the camera through the automatic defect annotation module to generate a defect annotation box.

[0007] Furthermore, the high dynamic range panoramic image library is established through the following steps: the drone, at the actual power line inspection altitude, uses a surrounding exposure method to cover each shooting point with a 360° full-view perspective to obtain a high dynamic range panoramic image. The original images taken by the drone are then combined with high dynamic range and 360° panoramic images. Based on the different locations and seasons in multiple provinces and cities in China, including mountains, fields, forests, wetlands, and cities, the image library is finely classified according to the shooting altitude to obtain the dynamic range panoramic image library.

[0008] Furthermore, the aforementioned conductor ground wire PBR material library is established through the following steps: using mature photogrammetry technology, 360° photographs are taken of real metal materials with different oxidation and corrosion levels to obtain material images. The illumination on the diffuse texture is removed and reconstructed into PBR materials. Seamless tiled material maps at various scales are created. The images are then finely classified according to the type and scale of the materials to obtain the conductor ground wire PBR material library.

[0009] Furthermore, the defect conductor / ground wire generation module is established through the following steps: a single-strand conductor model is predefined, which is generated by extending the cross-section of a single conductor / ground wire. The cross-section of the single-strand conductor model is a regular polygon. The single-strand conductor model is extended longitudinally by predefined control points. Each conductor / ground wire controls its direction, and the longitudinal conductor / ground wire grid vertices are interpolated by a spline interpolation algorithm. All single-strand conductor models arranged in the inner and outer circles are extended with different degrees of deflection to form a conductor / ground wire model. Based on this conductor / ground wire generation method, a control point at a random position near the complete conductor / ground wire model that does not coincide with the position of the conductor / ground wire model is defined as a broken strand control point. The position of each broken strand control point is constrained to a random position near the normal position, thereby generating defect conductor / ground wire generation modules with different shapes. Similarly, the generation method of defect conductor / ground wire generation modules with multiple broken strands can similarly generate multiple broken strand control points.

[0010] Furthermore, the aforementioned 3D rendering model of the defective conductor groundline is established through the following steps: To achieve a photorealistic rendering effect, this invention uses a 3D rendering engine based on the DirectX Raytracing (DXR) API (such as Unity3D, Unreal Engine) to render the image scene, but this invention is not limited to using the above rendering engines. The scene to be rendered is constructed as follows: First, a random observation point is selected and the camera's viewing angle is constrained to the direction in which the defect to be generated can be observed. A high dynamic range panoramic image is randomly selected from the high dynamic range panoramic image library according to the height of the camera's observation point as the background light source of the scene. The 3D model of the randomly broken conductor groundline defect is generated using the above method, and a material is randomly selected from the conductor groundline PBR material library and mapped onto the 3D rendering model of the defective conductor groundline, finally completing the ray tracing image rendering.

[0011] Furthermore, the automatic defect annotation module is established through the following steps: transforming the three-dimensional spatial coordinates of the bounding box of the broken strand model to the two-dimensional image coordinate system of the camera to generate a defect annotation box, and saving the annotation file at the same time as the image is rendered.

[0012] The beneficial effects of this invention are as follows: Based on three-dimensional image rendering technology and deep learning applications, this invention generates a large number of random samples of broken strand defects in UAV-inspected conductors with different backgrounds, lighting, and viewing angles and different strand shapes through a defect intelligent identification algorithm, providing a large amount of data support for the development of intelligent defect detection algorithms. Attached Figure Description

[0013] Figure 1 This is a structural block diagram of the sample generation system of the present invention;

[0014] Figure 2 This is a cross-sectional view of a real conductor;

[0015] Figure 3 Schematic diagram of longitudinal control point setting for a single-strand conductor model;

[0016] Figure 4 Schematic diagram of control point setting for conductor / ground wire broken strand defect model. Detailed Implementation

[0017] To further illustrate the purpose and technical solution of this invention, the invention will be described in more detail below with reference to the accompanying drawings and specific embodiments.

[0018] Reference Figure 1 As shown, the sample generation system upon which the defect sample generation of this invention relies runs on a 3D rendering server. It consists of a high dynamic range panoramic image library, a defect ground line generation module, a ground line PBR material library, an automatic defect annotation module, and a ray tracing-based defect ground line 3D rendering model. The sample generation process of this system involves randomly selecting a panoramic image from the high dynamic range panoramic image library as the light source, generating a defect ground line 3D rendering model through the defect ground line generation module, randomly selecting a metal material from the ground line PBR material library, generating a new seamless texture through multi-scale fusion, mapping the material onto the defect ground line 3D rendering model using UV mapping, generating an image using ray tracing progressive rendering, and converting the 3D spatial coordinates of the defect location bounding box to the 2D image coordinates of the camera through the automatic defect annotation module to generate a defect annotation box.

[0019] The specific steps of the method for generating defect samples of broken strands in overhead conductors by UAVs are as follows.

[0020] (1) Establishment of a high dynamic range panoramic image library.

[0021] In this invention, high dynamic range (HDR) panoramic images are captured by drones at actual power line inspection altitudes. Each shooting point uses ambient exposure and covers a 360-degree field of view. The original drone images are then combined using HDR and 360-degree panoramic techniques. The panoramic image library will collect images from various locations and seasons across multiple provinces and cities in China, including mountains, fields, forests, wetlands, and cities. The library will be finely categorized according to shooting altitude to obtain a dynamic range panoramic image library.

[0022] (2) Establishment of PBR material library for conductors and ground wires.

[0023] Mature photogrammetry technology is used to photograph and model various real metal materials with different degrees of oxidation and corrosion. The workflow of material photo modeling includes taking a 360-degree photo of the modeling object, removing the lighting on the diffuse texture, reconstructing it into a PBR material, creating seamless tiled material maps at various scales, and finely classifying the material library according to the type and scale of the material to obtain the conductor and ground wire PBR material library.

[0024] (3) Establishment of the defect conductor grounding generation module.

[0025] The single-strand conductor model is generated by extending a predefined cross-section, while the conductor-ground wire model is composed of single-strand models extended with varying degrees of deflection. Its extended mesh uses quadrilaterals, and the cross-section of the single-strand conductor-ground wire model is a regular polygon, such as... Figure 2 As shown on the left, the multi-strand cross-section is as follows Figure 2 The right figure shows that the longitudinal extension direction of each conductor is controlled by predefined control points, and the longitudinal conductor grid vertices are interpolated using a spline interpolation algorithm. The longitudinal control points for a single conductor are set as follows: Figure 3 As shown, control points at section a are sequentially set at sections b, c, d, etc., with a certain extension step length. The length of the step length directly controls the deflection of the conductor. The extension method of the inner conductor is set in the same way.

[0026] The creation of defects such as broken strands is based on this conductor / ground wire generation method. The control point at the broken strand location is defined at a random position near the complete conductor / ground wire model, but does not coincide with the position of the conductor / ground wire model. For example... Figure 4 As shown, each time a defect control point is generated, its position is constrained to a random location near the normal position, thus generating defect conductor generation modules with varying shapes. Similarly, the generation method for defect conductor generation modules with multiple broken conductors can similarly produce multiple defect control points.

[0027] (4) Establishment of a three-dimensional rendering model of the defective conductor ground wire.

[0028] To achieve a photorealistic rendering effect, this invention uses a 3D rendering engine based on the DirectX Raytracing (DXR) API (such as Unity3D, Unreal Engine) to render the image scene. However, this invention is not limited to using the above rendering engines. The scene to be rendered is constructed as follows: First, a random observation point is selected, and the camera's viewing angle is constrained to the direction in which the defect to be generated can be observed. Based on the height of the camera's observation point, a high dynamic range panoramic image is randomly selected from the high dynamic range panoramic image library as the background light source of the scene. The ground wire model of the random broken strand defect is generated using the above method, and a material is randomly selected from the material library and mapped onto the model. Finally, ray tracing image rendering is completed.

[0029] (5) Establishment of the automatic defect labeling module.

[0030] Transform the 3D spatial coordinates of the bounding box of the broken strand model to the camera image coordinate system to generate a label box, and save the label file at the same time as the image is rendered.

[0031] Those skilled in the art will readily understand that the above description is merely a preferred use case of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for generating defect samples of overhead conductor / ground wire broken strands during UAV inspection, characterized in that: The sample generation system includes a high dynamic range panoramic image library, a defect conductor ground line generation module, a conductor ground line PBR material library, an automatic defect annotation module, and a ray tracing-based 3D rendering model of the defect conductor ground line. This sample generation system runs on a 3D rendering server and includes the following steps: A panoramic image is randomly selected from the high dynamic range panoramic image library as the light source. A 3D rendering model of the defect ground line is generated through the defect ground line generation module. A new seamless texture is generated by randomly selecting a metal material from the ground line PBR material library and performing multi-scale fusion. The material is mapped onto the 3D rendering model of the defect ground line using the UV mapping method. Then, the image is generated using the ray tracing progressive rendering method. The 3D spatial coordinates of the defect part bounding box are converted to the 2D image coordinates of the camera through the automatic defect annotation module to generate the defect annotation box.

2. The method for generating defect samples of overhead conductor / ground wire broken strands by unmanned aerial vehicle (UAV) inspection according to claim 1, characterized in that, The high dynamic range panoramic image library is established through the following steps: Drones, at the actual height of power line inspections, use surrounding exposure to cover each shooting point with a 360° full-view perspective, capturing high dynamic range panoramic images. The original images are then combined with high dynamic range images and 360° panoramic images. Based on different locations and seasons in multiple provinces and cities across China, including mountains, fields, forests, wetlands, and cities, the images are finely categorized according to shooting height to obtain a high dynamic range panoramic image library.

3. The method for generating defect samples of overhead conductor / ground wire broken strands by unmanned aerial vehicle (UAV) inspection according to claim 2, characterized in that, The aforementioned PBR material library for ground wires is established through the following steps: We take 360° photos of real metal materials with different oxidation and corrosion levels for conductors and ground wires to obtain material images. We remove the lighting on the diffuse texture and reconstruct it into a PBR material. We then create seamless tiled material maps at various scales and classify the images according to the type and scale of the material to obtain a PBR material library for conductors and ground wires.

4. The method for generating defect samples of overhead conductor / ground wire broken strands by unmanned aerial vehicle (UAV) inspection according to claim 3, characterized in that, The aforementioned 3D rendering model of the defective conductor ground wire is obtained through the following steps: Define a single-strand conductor model generated by extending the cross section of a single conductor. The single-strand conductor model with a regular polygonal cross section is extended longitudinally by predefined control points. Each conductor controls its direction, and the longitudinal conductor grid vertices are interpolated by a spline interpolation algorithm. All single-strand conductor models arranged in the inner and outer circles are extended with different degrees of deflection to form a conductor model. Control points at random locations near the conductor-ground wire model that do not coincide with the location of the conductor-ground wire model are defined as strand break control points. The location of each strand break control point is constrained to a random location near the normal location, thereby generating 3D rendering models of defective conductor-ground wires with different shapes.

5. The method for generating defect samples of overhead conductor / ground wire broken strands by unmanned aerial vehicle (UAV) inspection according to claim 4, characterized in that, The aforementioned 3D rendering model of the defective conductor ground wire is established through the following steps: The image scene rendering is implemented using a 3D rendering engine based on the DirectX Raytracing API. The scene to be rendered is constructed as follows: First, an observation point is randomly selected and the camera observation angle is constrained to the direction in which the defect to be generated can be observed. A high dynamic range panoramic image is randomly selected from the high dynamic range panoramic image library according to the height of the camera observation point as the background light source of the scene. A 3D model of the conductor ground line of the random broken strand defect is generated. Then, a material is randomly selected from the conductor ground line PBR material library and mapped onto the 3D rendering model of the defect conductor ground line to complete the ray tracing image rendering.

6. The method for generating defect samples of overhead conductor / ground wire broken strands by unmanned aerial vehicle (UAV) inspection according to claim 5, characterized in that, The automatic defect annotation module performs defect annotation through the following steps: converting the three-dimensional spatial coordinates of the bounding box of the broken strand model to the two-dimensional image coordinates of the camera to generate a defect annotation box, and saving the annotation file at the same time as the image is rendered.

Images