Method, device and medium for detecting defects of a ground line

Abstract

Embodiments of the present application provide a method and device for detecting defects of a ground wire and a medium. The method comprises: acquiring a left view and a right view of a target region synchronously collected by a binocular imaging device; the target region includes a ground wire to be detected; based on the left view and the right view, a first depth map corresponding to the target region is acquired; pixel points outside a preset distance range in the first depth map are removed to obtain a second depth map; the left view, the right view and the second depth map are input into a pre-trained semantic segmentation model to obtain an image segmentation result; and based on the image segmentation result, a defect detection result of the ground wire to be detected is acquired. The method is used to improve the detection accuracy of defects of the ground wire.

Description

Technical Field

[0001] This application relates to the field of power systems, and in particular to a method, equipment and medium for detecting defects in conductors and ground wires. Background Technology

[0002] As the core carrier of the power system, the integrity of the conductors and ground wires of high-voltage transmission lines directly affects the stable operation of the power grid. With the rapid development of my country's ultra-high-voltage power grid, the coverage of transmission lines continues to expand, and the traditional manual inspection mode can no longer meet the needs of high-frequency and high-precision operation and maintenance.

[0003] Against this backdrop, intelligent inspection technology using drones equipped with optical imaging devices has become the mainstream solution. It uses sensors such as visible light and infrared to collect images of conductors and ground wires, and combines image processing algorithms to achieve defect detection.

[0004] However, due to the linear and slender physical characteristics of conductors and ground wires, they are difficult to extract accurately under complex backgrounds (such as blue sky, clouds, tree obstruction, and tower shadows), making it difficult to effectively detect defects on them. Therefore, there is an urgent need for a more accurate conductor and ground wire defect detection technology. Summary of the Invention

[0005] This application provides a method, device, and medium for detecting defects in conductors and ground wires, thereby improving the detection accuracy of defects in conductors and ground wires.

[0006] In a first aspect, embodiments of this application provide a method for detecting defects in a ground wire, including:

[0007] Acquire left and right views of a target area simultaneously captured by a binocular imaging device; the target area includes a ground wire to be detected.

[0008] Based on the left view and the right view, a first depth map corresponding to the target area is obtained; the pixel value of each pixel in the first depth map is the distance between the spatial position of the pixel in the target area and the shooting position.

[0009] Remove pixels in the first depth map that are outside a preset distance range to obtain the second depth map;

[0010] The left view, the right view, and the second depth map are input into a pre-trained semantic segmentation model to obtain image segmentation results; the image segmentation results include mask images of each defect region in the ground wire to be detected;

[0011] Based on the image segmentation results, the defect detection results of the ground wire to be detected are obtained; the defect detection results are used to indicate whether there are defect areas in the ground wire to be detected and the defect type of each defect area.

[0012] In one possible implementation, obtaining the first depth map corresponding to the target region based on the left view and the right view includes:

[0013] A semi-global block matching algorithm is used to obtain the disparity maps corresponding to the left view and the right view;

[0014] Based on the disparity map and the camera intrinsic parameters of the binocular imaging device, calculate the distance between the spatial position of each pixel in the disparity map in the target area and the shooting position;

[0015] The first depth map is constructed based on the distance corresponding to each pixel.

[0016] In one possible implementation, the semantic segmentation model is obtained by pre-training an improved YOLACT real-time instance segmentation model using a sample set based on a loss function; the sample set includes multiple labeled defective conductor samples and normal conductor samples.

[0017] The improved YOLACT real-time instance segmentation model is obtained by modifying at least one preset convolutional layer in the backbone network of the original YOLACT real-time instance segmentation model.

[0018] For each preset convolutional layer, the modification process involves adding a 1×N linear convolutional branch in parallel next to the 3×3 convolutional branch of the preset convolutional layer, and adding a weighted fusion layer based on an attention mechanism in series at the output ends of the 3×3 convolutional branch and the 1×N linear convolutional branch; where N is the preset convolutional length; the weighted fusion layer is used to assign weights to the output features of the two branches, and to perform weighted fusion of the output features of the two branches according to the assigned weights.

[0019] In one possible implementation, the method further includes:

[0020] For each defect region, the appearance features of the defect region are extracted based on the mask image of the defect region;

[0021] Based on the apparent characteristics, determine whether the defect region is a pseudo-defect;

[0022] Remove the defect areas that are identified as false defects in the defect detection results;

[0023] The apparent features include at least one of the following: texture features of the defect area, edge features, area features, and grayscale features.

[0024] In one possible implementation, the method further includes:

[0025] For each defect region, the actual size parameters of the defect region are calculated based on the camera intrinsic parameters of the binocular imaging device and the pixel value of each pixel in the second depth map.

[0026] Based on the actual size parameters and the preset grading standards, the severity level of the defect in the defective area is determined;

[0027] Add the defect severity level of each defect area to the defect detection results.

[0028] In one possible implementation, the method further includes:

[0029] Obtain the spatial coordinates of the shooting location;

[0030] Add the spatial coordinates to the defect detection result;

[0031] Based on the defect detection results of each conductor and ground wire to be inspected at various shooting positions during the inspection task, a conductor and ground wire inspection report is generated.

[0032] In one possible implementation, the method further includes:

[0033] For each defect detection result indicating a defective area in the inspection task, an alarm message is generated based on the defect detection result of the ground wire to be inspected.

[0034] The alarm information is pushed out.

[0035] Secondly, embodiments of this application provide a defect detection device for a ground wire, comprising:

[0036] The first acquisition module is used to acquire a left view and a right view of a target area synchronously acquired by a binocular imaging device; the target area includes a ground wire to be detected.

[0037] The second acquisition module is used to acquire a first depth map corresponding to the target area based on the left view and the right view; the pixel value of each pixel in the first depth map is the distance between the spatial position of the pixel in the target area and the shooting position.

[0038] The first rejection module is used to reject pixels in the first depth map that are outside a preset distance range to obtain a second depth map;

[0039] The processing module is used to input the left view, the right view, and the second depth map into a pre-trained semantic segmentation model to obtain image segmentation results; the image segmentation results include mask images of each defect region in the ground wire to be detected;

[0040] The third acquisition module is used to acquire the defect detection result of the ground wire to be detected based on the image segmentation result; the defect detection result is used to indicate whether there is a defect region in the ground wire to be detected and the defect type of each defect region.

[0041] In one possible implementation, the second acquisition module is specifically used for:

[0042] A semi-global block matching algorithm is used to obtain the disparity maps corresponding to the left view and the right view;

[0043] Based on the disparity map and the camera intrinsic parameters of the binocular imaging device, calculate the distance between the spatial position of each pixel in the disparity map in the target area and the shooting position;

[0044] The first depth map is constructed based on the distance corresponding to each pixel.

[0045] In one possible implementation, the semantic segmentation model in the processing module is obtained by pre-training an improved YOLACT real-time instance segmentation model using a sample set based on a loss function; the sample set includes multiple labeled defective conductor samples and normal conductor samples.

[0046] The improved YOLACT real-time instance segmentation model is obtained by modifying at least one preset convolutional layer in the backbone network of the original YOLACT real-time instance segmentation model.

[0047] For each preset convolutional layer, the modification process involves adding a 1×N linear convolutional branch in parallel next to the 3×3 convolutional branch of the preset convolutional layer, and adding a weighted fusion layer based on an attention mechanism in series at the output ends of the 3×3 convolutional branch and the 1×N linear convolutional branch; where N is the preset convolutional length; the weighted fusion layer is used to assign weights to the output features of the two branches, and to perform weighted fusion of the output features of the two branches according to the assigned weights.

[0048] In one possible implementation, the device further includes:

[0049] The second elimination module is used to extract the appearance features of each defect region based on the mask image of the defect region.

[0050] Based on the apparent characteristics, determine whether the defect region is a pseudo-defect;

[0051] Remove the defect areas that are identified as false defects in the defect detection results;

[0052] The apparent features include at least one of the following: texture features of the defect area, edge features, area features, and grayscale features.

[0053] In one possible implementation, the device further includes:

[0054] The determination module is used to calculate the actual size parameters of each defect region based on the camera intrinsic parameters of the binocular imaging device and the pixel value of each pixel in the second depth map.

[0055] Based on the actual size parameters and the preset grading standards, the severity level of the defect in the defective area is determined;

[0056] Add the defect severity level of each defect area to the defect detection results.

[0057] In one possible implementation, the device further includes:

[0058] The generation module is used to obtain the spatial coordinates of the shooting location;

[0059] Add the spatial coordinates to the defect detection result;

[0060] Based on the defect detection results of each conductor and ground wire to be inspected at various shooting positions during the inspection task, a conductor and ground wire inspection report is generated.

[0061] In one possible implementation, the device further includes:

[0062] The alarm module is used to generate alarm information based on the defect detection results of the ground wire to be tested in each defect detection result indicating a defective area during the inspection task.

[0063] The alarm information is pushed out.

[0064] Thirdly, embodiments of this application provide a processing device, including: a memory and a processor;

[0065] The memory stores computer-executed instructions;

[0066] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0067] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0068] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0069] The ground wire defect detection method, device, and medium provided in this application embodiment acquire left and right views simultaneously using a binocular camera to generate a first depth map. Then, using the actual shooting distance and the depth information of each pixel in the second depth image, the ground wire region in the image is filtered out to obtain a second depth map. Subsequently, by using the second depth map as an additional input to the semantic segmentation model, the model can combine the three-dimensional spatial information in the second depth map to perform image segmentation, which significantly reduces the interference of complex background on image segmentation and improves the segmentation accuracy of the model, thereby achieving the effect of improving the detection accuracy of ground wire defects. Attached Figure Description

[0070] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0071] Figure 1 A schematic flowchart illustrating a defect detection method for a ground wire provided in Embodiment 1 of this application;

[0072] Figure 2 This is a flowchart illustrating a method for detecting defects in a ground wire according to Embodiment 2 of this application.

[0073] Figure 3 This is a flowchart illustrating a method for detecting defects in a ground wire according to Embodiment 3 of this application.

[0074] Figure 4 This is a flowchart illustrating a method for detecting defects in a ground wire according to Embodiment 4 of this application.

[0075] Figure 5 This is a schematic diagram of the structure of a defect detection device for a ground wire provided in Embodiment 5 of this application;

[0076] Figure 6 This is a schematic diagram of the structure of a defect detection device for a ground wire provided in Embodiment Six of this application;

[0077] Figure 7 A schematic diagram of the processing device provided in this application.

[0078] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0079] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0080] To facilitate understanding of the technical content of this solution, the background technology is described in detail below:

[0081] As the coverage of high-voltage transmission lines expands, defects in conductors and ground wires, as core components of power transmission, such as broken strands or corrosion, can directly lead to line tripping or even tower collapse. Therefore, power facility maintenance personnel typically need to conduct defect inspections of each conductor and ground wire to ensure the stable operation of the power system.

[0082] Currently, due to the large inspection range, intelligent inspection technology using drones equipped with optical imaging equipment has become the mainstream solution. This mainstream solution uses visible light, infrared and other sensors to collect images of conductors and ground wires, and combines these images with image processing algorithms to achieve defect detection.

[0083] However, existing technical solutions still face a key bottleneck: due to the linear and slender physical characteristics of ground wires (typically <20mm in diameter), under interference from complex backgrounds (such as blue sky, clouds, tree obstructions, and tower shadows), existing segmentation algorithms struggle to accurately separate the ground wire from the background area, directly resulting in the inability to effectively detect defects on the ground wire. Therefore, a more accurate ground wire defect detection technology is urgently needed.

[0084] To address the problems mentioned in the background technology, the inventors discovered during their research that depth information of each pixel can be used to assist semantic segmentation: a depth map of the target area including the ground wire to be detected is obtained from an image acquired by a binocular imaging device. The actual shooting distance is used to perform preliminary background removal processing on each pixel in the depth map, retaining only the local area covering the ground wire. Then, the depth map after background removal is used to assist semantic segmentation, which can accurately separate the ground wire from the complex background. At the same time, compared with traditional segmentation based solely on visual features, the introduction of depth information can also improve the segmentation accuracy of the model for defective areas in scenarios such as backlighting, thereby effectively improving the accuracy of ground wire defect identification.

[0085] It should be noted that this application is applicable to the defect detection scenario of conductors and ground wires in high-voltage transmission lines. The method provided in this application can be directly executed in the embedded processing unit of a UAV, or it can be executed by a ground processing terminal, cloud server or other equipment with data processing capabilities using the conductor and ground wire related images collected and transmitted back by the UAV. This application does not impose specific restrictions on the subject executing the method.

[0086] As a specific scenario example, the solution provided in this application can be implemented in a multi-rotor drone. Specifically, the multi-rotor drone includes a binocular imaging device and an embedded processing unit (e.g., an edge computing module of NVIDIA Jetson Xavier NX) connected via a Peripheral Component Interconnect Express (PCIe) interface. The drone uses the binocular imaging device to acquire dual views of the ground wire to be inspected at a fixed point, and executes the defect detection method for the ground wire provided in this application through the embedded processing unit to obtain the defect detection result of the ground wire. Optionally, the drone also includes a communication module connected to the embedded processing unit via a Universal Serial Bus (USB) for transmitting the acquired defect detection result back to the ground processing terminal for maintenance personnel to view and analyze; to avoid signal obstruction by the fuselage, the antenna used for communication can be placed on the outside of the fuselage. Optionally, the drone may also include a positioning module connected to the embedded processing unit via a Universal Asynchronous Receiver / Transmitter (UART) to obtain the spatial coordinates of the shooting location, such as a UBLOX ZED-F9P Global Navigation Satellite System positioning module.

[0087] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0088] Figure 1 This is a flowchart illustrating a method for detecting defects in a ground wire according to Embodiment 1 of this application. Figure 1 As shown, the method provided in this embodiment includes:

[0089] S101. Obtain the left and right views of the target area synchronously acquired by the binocular imaging device.

[0090] In this step, in order to acquire image and depth information of the conductor to be inspected, it is necessary to obtain dual views (left and right views) of the target area including the conductor to be inspected. In practical applications, the left and right views of the target area can be simultaneously acquired by a drone at a pre-set shooting location using its onboard binocular imaging device.

[0091] Specifically, a binocular imaging device refers to a device configured to include two cameras with fixed relative positions and orientations, capable of simultaneously acquiring image data of the same target area from different perspectives; correspondingly, the left view and right view refer to the two-dimensional images of the target area simultaneously acquired by the left and right cameras in the binocular imaging device, respectively.

[0092] It should be understood that, considering the considerable length of some ground wires, a single shooting location may not be sufficient to completely cover the entire length of the ground wire and capture defects in all sections. Therefore, multiple shooting locations can be set up for the same section of ground wire to be inspected, with each shooting location corresponding to a target area that covers different sections of the ground wire. Thus, for longer ground wires, the defect detection results determined based on images acquired in the target area in this solution typically correspond to a specific section of the ground wire. Accordingly, for ease of reference, "ground wire to be inspected" in this application will refer to a specific section of the ground wire to be inspected. The detection results for the complete section of the ground wire can be obtained by summarizing the defect detection results corresponding to each determined section.

[0093] Regarding the shooting location, it can be selected within a range of 5-8m horizontally and 3-5m vertically from the grounding wire. Additionally, to ensure accurate acquisition of dual views of the target area including the grounding wire, the shooting angle of the binocular imaging device needs to be set. For example, the angle between the shooting angle and the axis of the grounding wire can be set to ≤15° to ensure a stable proportion of the grounding wire in the image. In practical applications, the gimbal control accuracy of the drone should be ≤±0.01° to avoid blurry images caused by camera shake, which would affect the accuracy of subsequent image segmentation.

[0094] As a specific application scenario, starting from a position 6m horizontally and 4m vertically away from one end of the ground wire to be inspected, a drone can be controlled to fly horizontally along the ground wire at a speed of less than 5m / s. A binocular imaging device can be controlled to acquire images at a preset frame rate (e.g., 30fps), generating a unique timestamp for each frame, until the drone reaches the other end of the ground wire. Based on this, the defect detection method for the ground wire provided in this application can be applied to both the left and right views corresponding to each timestamp, thus achieving defect detection results for the complete segment of the ground wire to be inspected.

[0095] S102. Based on the left and right views, obtain the first depth map corresponding to the target area.

[0096] In the first depth map, the pixel value of each pixel is the distance between the pixel's spatial location in the target area and the shooting location. It should be understood that each pixel value in this image directly reflects the depth information of the corresponding spatial point.

[0097] In this step, based on the acquired left and right views, a stereo matching algorithm based on local matching can be used to obtain a disparity map. For example, the horizontal displacement between each pixel in the left view and its corresponding matching point in the right view can be calculated to obtain the disparity value. Subsequently, the disparity value of each pixel can be converted into the distance between its corresponding spatial position in the target area and the shooting position according to the principle of triangulation. Finally, the first depth map is constructed based on the distances of each pixel.

[0098] In one possible implementation, this step can be achieved using steps 2.1 to 2.3 as follows:

[0099] Step 2.1: Use the Semi-Global Block Matching (SGBM) algorithm to obtain the disparity maps corresponding to the left and right views.

[0100] Among them, the SGBM algorithm is a disparity calculation method widely used in the field of binocular vision. Its core lies in the global optimization of local matching costs by performing one-dimensional path aggregation in multiple directions, which has high processing efficiency and computational accuracy.

[0101] In this step, the disparity maps corresponding to the left and right views will be calculated using the SGBM algorithm according to the pre-configured parameter set. The pixel value of each pixel in this disparity map represents the magnitude of the disparity between the left and right views.

[0102] The parameter set includes, for example, the matching window size, disparity search range, and smoothness penalty coefficient for matching calculation. The smoothness penalty coefficient includes a penalty P1 for small disparity changes and a penalty P2 for drastic disparity changes. For example, the parameters might be configured as follows: window size = 9×9, minimum disparity = 0, maximum disparity = 64, P1 = 8×3×3², P2 = 32×3×3².

[0103] Optionally, to further improve the quality of the disparity map and provide a more accurate data foundation for subsequent depth map extraction, image optimization processing can be performed on the acquired left and right views before this step. This image optimization processing may include, for example, Gaussian filtering and image noise removal.

[0104] Step 2.2: Based on the disparity map and the camera intrinsic parameters of the binocular imaging device, calculate the distance between the spatial position of each pixel in the target area and the shooting position in the disparity map.

[0105] Among them, the camera intrinsic parameters include the focal length f and baseline b of the binocular imaging device.

[0106] Specifically, for each pixel in the disparity map, the distance (i.e., depth) between the spatial position of the pixel in the target area and the shooting position can be calculated using the formula Z=(f×b) / d based on the disparity value d of that pixel.

[0107] Optionally, to further improve the quality of the depth map and provide more accurate depth information for subsequent image segmentation, the acquired disparity map can be optimized before this step. Optimization processing may include at least one of disparity consistency detection, hole filling, median filtering, and disparity noise removal.

[0108] In detail, disparity consistency detection refers to calculating the left and right disparity maps based on the left and right views respectively, performing bidirectional verification of the disparity values ​​at the same pixel position, and eliminating erroneous matching points that do not meet the consistency constraints; hole filling refers to interpolating and filling invalid or empty areas in the disparity map using the effective disparity values ​​of the neighborhood to ensure the continuity of the disparity map; median filtering refers to replacing the original pixel value with the median disparity value in the neighborhood window centered on the current pixel, suppressing salt-and-pepper noise and preserving disparity edge information; disparity noise removal refers to eliminating abnormal disparity values ​​caused by missing image textures and matching jitter through smoothing constraints or filtering operations, thereby improving the overall reliability of the disparity map.

[0109] Step 2.3: Construct the first depth map based on the distance corresponding to each pixel.

[0110] In this step, the distance corresponding to each pixel will be used as the value of the pixel to construct a depth map with the same size as the disparity map, thus obtaining the first depth map.

[0111] The method provided in steps 2.1 to 2.3 of this implementation uses a semi-global block matching algorithm to calculate the disparity map. This algorithm's high robustness effectively improves the disparity calculation accuracy in outdoor linear object extraction scenarios like this one. Furthermore, its high efficiency ensures efficient disparity calculation, guaranteeing the detection efficiency of ground wires. Based on this, the actual distance of each pixel is calculated using the disparity map and the camera intrinsics of the binocular imaging device. The disparity values ​​are then converted into physical distances using the camera intrinsics, ensuring the authenticity and applicability of the depth information. Finally, a first depth map is constructed based on the distance corresponding to each pixel, forming a structured depth data representation that provides a usable data foundation for subsequent semantic segmentation.

[0112] S103. Remove pixels in the first depth map that are outside the preset distance range to obtain the second depth map.

[0113] In this step, to reduce the interference of distant background elements such as the sky and nearby tree branches on the segmentation of the ground plane region, pixels outside a preset distance range in the first depth map are removed to obtain the second depth map. Furthermore, this removal process also ensures that the subsequent semantic segmentation process has a smaller computational load.

[0114] The preset distance range is determined based on the target distance between the shooting location and the conductor to be inspected. The preset distance range should include the target distance.

[0115] For example, assuming the shooting location is 6.5m away from the conductor to be detected, the preset distance range can be 5~8m. Accordingly, pixels in the first depth map with distances Z>8m and Z<5m can be marked as invalid.

[0116] It should be understood that using the second depth map as input to the subsequent semantic segmentation model can increase the model's attention to local regions that retain depth information, thereby improving the segmentation accuracy of conductors and ground lines covered within those regions.

[0117] S104. Input the left view, right view, and second depth map into the pre-trained semantic segmentation model to obtain the image segmentation result.

[0118] The image segmentation results include a mask image of each defect region in the conductor to be detected.

[0119] In this step, the left view, right view, and depth map need to be stitched together to form a 3-channel feature map, which is used as the input to the semantic segmentation model. This allows the model to perform semantic segmentation based on the input data and pre-trained model parameters, obtaining mask images of each defect region in the conductor.

[0120] In practical applications, image segmentation results typically include a mask image of the conductor to be detected. Additionally, the image segmentation results include classification labels for each defect region to facilitate determining the category of each defect region based on the segmentation results. For example, in the image segmentation results, mask images corresponding to different types of defect regions are distinguished by different colors. If the defect types include broken wire strands, line corrosion, and foreign matter attached to the line, then in the image segmentation results, broken wire strands can correspond to a red mask, line corrosion to a green mask, and foreign matter attached to the line to a blue mask.

[0121] It should be understood that using different colors to mark different types of defects facilitates the determination of the type of each defect region present in the conductor to be detected based on the image segmentation results.

[0122] The semantic segmentation module is used to divide different semantic regions based on the input data. It can be obtained by training an initial model based on a classic semantic segmentation network using a sample set. Such semantic segmentation network includes, but is not limited to, fully convolutional networks (FCN), U-Net, and segmentation networks (SegNet). This application does not specify the structure type of the initial model.

[0123] The sample set includes multiple labeled defective conductor samples and normal conductor samples.

[0124] As a specific example, to improve the robustness and segmentation accuracy of the trained semantic segmentation model, obtaining the sample set may include the following steps a and b:

[0125] Step a: Collect ground wire image data (left view and right view) under different scenarios. The scenario can be limited according to the shooting lighting conditions, background interference, and the defects, voltage level and material of the ground wire.

[0126] Lighting conditions include, for example, sunny days (e.g., shooting time 9:00-11:00, light intensity 8000-12000 lux), cloudy days (e.g., light intensity 2000-5000 lux), backlighting (e.g., shooting time 15:00-17:00, sun incidence angle ≤30°), and evening (e.g., light intensity 500-1000 lux); background interference includes, for example, foliage obstruction (partial / complete obstruction), tower shadows, foggy weather, and bird interference; defect types include, for example, normal defects, broken strands, line corrosion, and foreign objects attached to the line, where broken strands can be further subdivided into single wire breaks (e.g., broken strand length 3-10mm) and multiple wire breaks (e.g., broken strand length 10-20mm), and line corrosion can be further subdivided into pitting corrosion (e.g., corrosion area 5-10mm). 2 ) and surface corrosion (e.g., corrosion area of ​​10-20mm) 2 Foreign objects attached to the line can be further specified as plastic film wrapping (e.g., foreign object length of 50-100mm) and tree branch overlap (e.g., foreign object length of 5-10mm); voltage levels include 110kV, 200kV and 500kV; materials include steel-cored aluminum stranded wire and aluminum alloy stranded wire, etc.

[0127] The number of frames captured for each scene can be determined based on the actual model detection accuracy requirements. For example, at least 1500 frames can be captured for each scene, ensuring that images with defects account for at least 40% of the total images to meet the detection requirements for samples with minute defects, and ensuring that images under backlight conditions account for at least 20% of the total images to meet the detection requirements for complex scenes.

[0128] Optionally, to enrich the training set, data augmentation processing such as random horizontal flipping, brightness and contrast adjustment can be performed on the acquired images.

[0129] Step b: Label the collected conductor and ground wire image data to construct a sample set.

[0130] Firstly, regarding the annotation methods, for example: trained annotators should use annotation tools (such as LabelMe 5.0) to annotate each left view according to a unified standard, following the principle of "segmenting the region first, then annotating the defect." Specifically, two levels of labels (first-level labels and second-level labels) are required. The first-level label annotation method uses closed polygons to annotate the "conductor / ground line region" and the "background region," ensuring that the annotation error of the conductor / ground line edge is ≤1 pixel, for the model to learn the overall segmentation of the conductor / ground line. The second-level label annotation method uses polygons to further annotate "normal segment," "broken segment," "corroded segment," and "foreign object segment" within the conductor / ground line region. Broken segments are annotated with the fracture endpoints, and corroded segments are annotated with the corrosion boundaries, for the model to learn the fine segmentation of different defects.

[0131] Optionally, to ensure the accuracy and reliability of sample annotation, a quality control mechanism based on consistency verification can be implemented on the annotated sample set: a preset percentage (e.g., 10%) is randomly selected from the completed annotated samples as verification samples, and two annotators independently annotate them; the Cohen's Kappa coefficient is used to quantitatively evaluate the consistency of the results of the two annotators. If the consistency coefficient meets the preset threshold, the annotation is deemed qualified; otherwise, the sample annotation is deemed unqualified and returned for re-annotation. This ensures the quality of the dataset annotation and provides reliable data support for the training of the semantic segmentation model.

[0132] Next, each annotated left view, its corresponding right view at the same timestamp, and the second depth map obtained based on the left and right views are used as samples in the sample set to form a sample set. Samples indicating "normal" defects in the image data are used as normal conductor samples in the sample set; samples indicating "non-normal" defects in the image data are used as defective conductor samples in the sample set.

[0133] Step c: Divide the sample set into training, validation, and test sets according to a preset ratio. It should be understood that the proportion of image data for each scene should be consistent across the three groups to avoid data bias and affecting model training accuracy. For example, the preset ratio could be 70% training set, 20% validation set, and 20% test set.

[0134] The training set is used to train the semantic segmentation model. By inputting labeled image samples (including left view, right view, depth map, and corresponding mask labels), the model learns the feature patterns of ground wire regions and various defect regions, and optimizes model parameters to achieve accurate segmentation. The validation set is used to verify the model's training effect and adjust hyperparameters. During the model training process, the model's segmentation accuracy is evaluated, and hyperparameters such as learning rate and batch size are optimized based on the validation results to avoid overfitting or underfitting. The test set is used to finally evaluate the model's generalization ability and actual segmentation performance. It uses brand-new samples that have not participated in model training and parameter adjustment to test the model's segmentation effect on unknown data, verifying whether the model can stably and accurately identify ground wires and various defects, and ensuring that the model meets the actual requirements for ground wire defect detection.

[0135] By using the sample set constructed through the above image acquisition, standardized annotation, and set partitioning methods to train the selected initial model, it can be ensured that the trained semantic segmentation model can output image segmentation results that meet the requirements of this scheme.

[0136] In one possible implementation, to further improve the model's accuracy in extracting ground wires, the semantic segmentation model can be a pre-trained improved YOLACT real-time instance segmentation model based on a loss function and a sample set.

[0137] Specifically, the improved YOLACT (You Only Look At Coefficients) real-time instance segmentation model is based on the original YOLACT model with structural or algorithmic optimizations to adapt to the specific task requirements of conductor and ground wire defect detection.

[0138] In detail, the improved YOLACT real-time instance segmentation model is obtained by modifying at least one pre-defined convolutional layer in the backbone network of the original YOLACT real-time instance segmentation model.

[0139] The backbone network is the basic network structure in the model used to extract image features. It is responsible for extracting multi-scale feature maps with semantic information from the original input image. For example, it can be ResNet-50, Darknet, or EfficientNet. The pre-specified convolutional layers are convolutional layers in the backbone network that are pre-specified or selected for modification processing. These layers are usually located in the middle of the backbone network, such as the 3rd to 5th layers of the backbone network.

[0140] The specific modifications to the preset convolutional layers are as follows: For each preset convolutional layer, add a 1×N linear convolutional branch in parallel next to the 3×3 convolutional branch of the preset convolutional layer, and add a weighted fusion layer based on the attention mechanism in series at the output of the 3×3 convolutional branch and the 1×N linear convolutional branch.

[0141] Among them, the 1×N linear convolution branch uses a 1-row N-column (or N-row 1-column) elongated convolution kernel for feature extraction. This type of linear convolution kernel is particularly suitable for capturing linear structures or elongated features in images. Specifically, N is the preset convolution length, which can be set according to the width, length and other characteristics of the ground line. For example, N can be 5, 7, 9, etc.

[0142] The weighted fusion layer is used to assign weights to the output features of the two branches and then perform weighted fusion of the output features of the two branches according to the assigned weights. It should be understood that the attention mechanism allows the model to dynamically focus on the more important parts of the input data and suppress the less important parts when processing information. This can be achieved, for example, through channel attention or spatial attention, thereby obtaining more discriminative fused features.

[0143] As a concrete example, in the improved YOLACT real-time instance segmentation model, each layer from layers 3 to 5 of the backbone network includes a parallel 3×3 convolutional branch and a 1×7 linear convolutional branch, followed by a weighted fusion layer. During the training and application phases of this model, the weighted fusion layer determines the corresponding weight coefficients of the two branches based on the feature response values ​​of the linear convolutional branches to the ground line. Specifically, firstly, global average pooling is performed on the output feature map of the 1×7 linear convolutional branch to obtain the linear feature response value of the ground line. This response value is then fed into the weighted fusion module, where a fully connected layer and activation function automatically generate channel-level adaptive weight coefficients, which are positively correlated with the linear feature response value. Subsequently, these weights are applied to the features of the 1×7 linear convolutional branch, and complementary weights are applied to the features of the 3×3 convolutional branch. The weighted fusion of the two branches is completed through element-wise addition, achieving dynamic automatic adjustment of the weight coefficients and enhancing the model's ability to capture features of slender targets like the ground line.

[0144] In addition, the loss function used for training is a function that measures the difference between the model's prediction and the true label. Its output value is used to guide the model to adjust parameters during training in order to minimize the prediction error. For example, cross-entropy loss, Dice loss, or focus loss functions can be used.

[0145] The focus loss function is a loss function specifically designed to address the class imbalance problem in object detection and semantic segmentation. It modifies the standard cross-entropy loss function by reducing the weight of the loss on easily classified samples, thus allowing the model to focus more on learning difficult-to-classify samples.

[0146] In detail, the focusing loss function can be expressed as:

[0147]

[0148] Where N represents the total pixel data of a single image; t represents the true labeling category of the pixel (such as defective pixels or normal pixels). This represents the class weight coefficient corresponding to the true labeled class t of the i-th pixel. This represents the predicted probability of the i-th pixel in its true labeled class t;

[0149] It should be understood that the inclusion of class weight coefficients is intended to enable the loss function to assign different weight values ​​to different types of pixels based on their class: assigning larger class weight coefficients to normal pixels and smaller class weight coefficients to defective pixels. This configuration of the loss function allows the model to focus more on those difficult-to-distinguish defective regions, effectively addressing the problem of poor model training accuracy caused by the small number of defective pixels and the imbalanced class distribution in this application.

[0150] The semantic segmentation model acquisition method provided in this implementation pre-trains the improved YOLACT real-time instance segmentation model using a sample set containing both defective and normal ground wire samples, based on a loss function. This ensures the model can fully learn the specific morphology and defect patterns of ground wires, thereby improving its generalization ability. Furthermore, by modifying the pre-defined convolutional layers in the backbone network of the original YOLACT real-time instance segmentation model—specifically by adding a 1×N linear convolutional branch in parallel alongside the 3×3 convolutional branch and connecting it to a weighted fusion layer based on an attention mechanism—the model can specifically extract features from the linear characteristics of ground wires. The 1×N linear convolutional branch effectively captures slender structures, avoiding the omission of features from slender objects in complex backgrounds that traditional convolutions often fail to capture. Simultaneously, the weighted fusion layer based on an attention mechanism dynamically assigns weights and weights the output features of the two branches, adaptively strengthening relevant linear features and suppressing irrelevant background interference, ensuring the model accurately focuses on ground wire features even in complex environments. Overall, this improvement enables the semantic segmentation model to more accurately identify and segment defective regions of ground wires, significantly improving the accuracy of image segmentation results.

[0151] It should be noted that, in practical applications, the pre-trained semantic segmentation model provided in this application is typically trained on devices with significant computing power, such as ground processing terminals. Then, the model with the best performance (e.g., the model's Intersection over Union (IoU) for conductor-to-ground wire segmentation and the F1 score for defect detection can be used to evaluate the performance of each model) is determined as the final semantic segmentation model and deployed on the execution device (such as a UAV) of the conductor-to-ground wire defect detection method of this application, so that the execution device can use the deployed model to perform defect detection. Optionally, during model training, gradient descent optimization can be used to avoid overfitting.

[0152] Alternatively, in practical applications, to facilitate model processing and improve model segmentation efficiency, the left view, right view, and second depth map can all be converted into grayscale images, and then their corresponding grayscale images can be used as input to the model. This can be achieved by normalizing the pixel values ​​of each pixel to 0-225.

[0153] S105. Based on the image segmentation results, obtain the defect detection results of the ground wire to be detected.

[0154] Among them, the defect detection results are used to indicate whether there are defective areas in the conductor to be tested and the type of defect in each defective area.

[0155] In this step, if the image segmentation result contains a mask image of a defective region, it is determined that the ground wire to be detected has a defective region; otherwise, it is determined that the ground wire to be detected does not have a defective region. When the ground wire to be detected has a defect, the corresponding defect type can be determined directly based on the classification identifier (such as mask color) of each defective region in the image segmentation result.

[0156] The ground wire defect detection method provided in this application embodiment acquires left and right views simultaneously using a binocular camera and generates a first depth map. Then, using the actual shooting distance and the depth information of each pixel in the second depth image, the ground wire region in the image is filtered out to obtain a second depth map. Subsequently, by using the second depth map as an additional input to the semantic segmentation model, the model can combine the three-dimensional spatial information in the second depth map to perform image segmentation, which significantly reduces the interference of complex background on image segmentation and improves the segmentation accuracy of the model, thereby achieving the effect of improving the detection accuracy of ground wire defects.

[0157] Furthermore, Figure 2 This is a flowchart illustrating a method for detecting defects in a ground wire according to Embodiment 2 of this application. Figure 2As shown, to further improve the detection accuracy of the conductor and ground wire, based on the above embodiments, the method provided in this embodiment further processes the aforementioned defect detection results, specifically including:

[0158] S201. For each defect region, extract the appearance features of the defect region based on the mask image of the defect region.

[0159] In this step, to further distinguish whether the defect regions indicated in the defect detection results are real defect regions, the appearance features of the corresponding defect regions will be extracted based on the mask images of each defect region. Specifically, for each defect region, the location of the defect in the grayscale image corresponding to the defect in the left view can be located first using the mask image of the defect region, and then the appearance features of the corresponding location region can be extracted to obtain the appearance features of the defect region.

[0160] Among them, appearance features refer to indicators that can describe the visual attributes of the defective area, including at least one of the texture features, edge features, area features and grayscale features of the defective area.

[0161] Optionally, before using the mask image to determine the location of the defect in the left view, the mask of the defect region can be optimized to improve the accuracy of the target region determination. This optimization process may include, for example, adjusting for tiny holes in the defect region through morphological dilation.

[0162] In some implementations, for each defect area, a contour detection algorithm can be used to extract the bounding rectangle of the corresponding location area in the grayscale image of the left view, thus obtaining the grayscale rectangular area corresponding to the defect area. Then, features can be extracted from the grayscale rectangular area from dimensions such as texture, edge, area, and grayscale to obtain the appearance features corresponding to the defect area.

[0163] Optionally, for each gray-scale rectangular region, the texture information of the region can be quantified using methods such as Gabor filter, Local Binary Pattern (LBP), or Gray Level Co-occurrence Matrix (GLCM) to extract the texture features of the defect region; edge detection algorithms such as Canny operator or Laplacian operator can be used to identify the edge discontinuity of the defect in the region to extract edge features; the area of ​​the connected region of the defect region in the pixel plane can be directly calculated to obtain the area features; and the average gray value, gray-scale histogram, gray-scale variance, and other statistics of the defect region in the region can be calculated to obtain the gray-scale features.

[0164] S202. Based on the appearance characteristics, determine whether the defect area is a pseudo-defect.

[0165] In this step, for each defective area, the apparent features of the defective area need to be compared with the features of a normal conductor / ground wire. Based on the degree to which it deviates from the features of a normal conductor / ground wire, it is determined whether it is a false defect. Alternatively, it can be determined whether the apparent features of the defective area meet the preset defect judgment conditions.

[0166] For example, the apparent features of the defective region include: the LBP histogram of the defective region in the gray-scale rectangular region, the edge discontinuity, the area of ​​the region, and the gray-scale difference between the defective region and the surrounding normal conductors; correspondingly, when the cosine similarity between the LBP histogram of the defective region and the LBP histogram of the normal conductor is greater than a preset similarity threshold (e.g., 0.9), the edge discontinuity of the defective region is less than a preset continuity threshold (e.g., 0.15), the gray-scale difference of the defective region is less than a preset gray-scale difference threshold (e.g., 10), and the area of ​​the defective region is less than a preset pixel threshold (e.g., 5 pixels), the defective region is determined to be a pseudo-defect.

[0167] S203. Remove defect areas that are identified as false defects in the defect detection results.

[0168] In this step, the defect areas marked as false defects in the defect detection results will be directly deleted to obtain the final defect detection results for the conductor to be tested.

[0169] The conductor and ground wire defect detection method provided in this application significantly improves the accuracy and reliability of conductor and ground wire defect detection by extracting the apparent features of each defect area and identifying and eliminating false defects based on these features. This results in more accurate final defect detection results and provides a more reliable basis for subsequent line maintenance and decision-making.

[0170] Figure 3 This is a flowchart illustrating a method for detecting defects in a ground wire according to Embodiment 3 of this application. Figure 3 As shown, to further improve the convenience of information acquisition for maintenance personnel and thus improve the maintenance efficiency of conductors and ground wires, based on the above embodiments, the method provided in this implementation further processes the aforementioned defect detection results, specifically including:

[0171] S301. For each defect region in the defect detection result, calculate the actual size parameters of the defect region based on the camera intrinsic parameters of the binocular imaging device and the pixel value of each pixel in the second depth map.

[0172] The defect detection result can be the defect detection result determined directly from the model output in step S105, or the defect detection result obtained after the elimination process in step S203. This application does not impose any restrictions on this.

[0173] In this step, in order to convert the defect region obtained from image segmentation from pixel size to actual size parameters, it is also necessary to calculate the actual size parameters of the defect region in the image (such as the mask image of the defect region) for each defect region using a second depth map that records depth information and the camera parameters of the binocular imaging device, based on principles such as triangulation.

[0174] The actual dimensional parameters include, for example, the length of the broken strand area, the corrosion depth and area of ​​the corrosion zone on the line, and the length and width of the area on the line where foreign matter is attached.

[0175] For example, the actual dimensional parameters such as the length, width, and area of ​​each defect region can be calculated using the following formula:

[0176] Actual size = (number of pixels × s × Z) / f, where s represents the single pixel size of the binocular imaging device, and f represents the focal length of the binocular imaging device. For example, if the focal length f = 16mm and the single pixel size s = 3.4μm, then a single pixel with a depth Z of 6m corresponds to an actual size of 0.127mm in space.

[0177] Therefore, the actual size parameters of the defect region can be calculated directly based on its depth and the number of pixels. For example, for a defect region with an average depth of 6m, its actual physical length can be obtained by multiplying the total number of pixels in the horizontal direction of the defect region by 0.127, and its area can be obtained by multiplying the total number of pixels in the defect region by the square of 0.127.

[0178] Another example is that the actual size parameter of the corrosion zone depth can be calculated by the following method: calculate the difference ΔZ between the average depth of the conductor and the maximum depth of the corrosion zone; use the formula ΔZ⊥=ΔZ×cosθ to perform a correction process based on the shooting angle on the difference to obtain the actual size parameter ΔZ⊥ of the depth, where θ represents the pitch angle of the camera.

[0179] S302. Determine the severity level of the defect area based on the actual dimensional parameters and the preset grading standards.

[0180] In this step, for each defect area, the severity of the defect needs to be quantified based on the actual size parameters of the defect area and the preset grading standards.

[0181] In one possible implementation, the grading criteria include multiple size thresholds for classifying grades. Accordingly, the severity level of a defective region is determined, for example, by classifying a defective region of type "broken strand" as a minor defect if its actual length is less than L1, a moderate defect if it falls between L1 and L2, and a severe defect if it exceeds L2. It should be understood that, similarly, the severity level of a defective region of a given type can also be determined based on corresponding size thresholds set for area and depth for other types of defective regions.

[0182] The various size thresholds set in the grading standards can be based on industry standards, historical data, or expert experience.

[0183] As a specific example, the defect classification can be based on DL / T741-2010 "Operation Regulations for Overhead Transmission Lines" and the actual dimensional parameters of each defect area:

[0184] For example, regarding areas with broken strands, if the length of the defective area is less than 5mm, the defect level is determined to be a minor defect; if the length is 5-10mm, the defect level is determined to be a moderate defect; and if the length is greater than 10mm, the defect level is determined to be a severe defect. Similarly, regarding areas of line corrosion, if the corrosion area is less than 10mm... 2 If the corrosion depth is less than 0.2 mm, the defect level is determined to be a minor defect; if the corrosion area is 10-30 mm... 2 If the corrosion depth is 0.2-0.5 mm, the defect level is determined to be moderate; if the corrosion area is >30 mm... 2 If the corrosion depth is greater than 0.5 mm, the defect level is determined to be severe. For areas with foreign matter attached to the line, if the length of the defect area is less than 50 mm, the defect level is determined to be mild; if the length of the defect area is 50-100 mm, the defect level is determined to be moderate; if the length of the defect area is greater than 100 mm, the defect level is determined to be severe.

[0185] In another possible implementation, multiple actual size parameters of the defective region (such as length, width, and area) and other possible features (such as shape complexity and material) can be combined to make a comprehensive judgment using a fuzzy logic system or a pre-trained machine learning model (such as support vector machine, decision tree, etc.) and output a discrete level classification.

[0186] S303. Add the defect severity level of each defect area to the defect detection results.

[0187] In this step, in order to increase the information dimensions of the defect detection results so that subsequent maintenance personnel can obtain more comprehensive evidence based on the defect detection results, it is necessary to add the defect severity level to the data item of the corresponding defect area in the defect detection results.

[0188] For example, in the data structure of the defect detection results, a field can be added for each identified defect area to store its corresponding severity level (e.g., the string "minor", "moderate", "severe" or the number 1, 2, 3, etc.).

[0189] The defect detection method for conductors and ground wires provided in this application calculates the actual size parameters based on camera parameters and quantifies the severity of defects based on the actual size parameters of each area. Finally, the determined severity level is added to the defect detection results. This allows maintenance personnel to directly understand the severity of defects based on the defect detection results when conducting on-site calculations and assessments. This provides more timely, accurate, and comprehensive data support for subsequent maintenance plan formulation, resource allocation, and risk assessment, effectively improving the intelligence, refinement, and efficiency of conductor and ground wire maintenance.

[0190] Furthermore, Figure 4 This is a flowchart illustrating a method for detecting defects in a ground wire according to Embodiment 4 of this application. Figure 4 As shown, based on the above embodiments, this embodiment provides a conductor / ground wire defect detection method for the entire inspection task, including:

[0191] S401. For each conductor to be inspected in the inspection task, acquire the left and right views of the target area synchronously collected by the binocular imaging device, as well as the spatial coordinates of the shooting position.

[0192] The target area includes the ground wire to be detected.

[0193] In practical applications, a single inspection activity of a drone typically involves inspecting for defects in one or more ground wires to be inspected. Therefore, in this step, for each ground wire to be inspected in the inspection task, the left and right views of the ground wire to be inspected will be obtained at the corresponding shooting position; at the same time, in order to obtain the spatial coordinates of the ground wire to be inspected so that maintenance personnel can clearly identify the geographical location of the defect, the spatial coordinates of the shooting position will also be obtained simultaneously.

[0194] The spatial coordinates include the latitude and longitude coordinates of the shooting location, and optionally, the altitude information of the shooting location. In a specific implementation, the spatial coordinates of the shooting location can be determined by a positioning module mounted on the drone. For example, the positioning module outputs positioning data at a fixed sampling frequency (e.g., 10Hz). If the drone's image sampling frequency is 30fps, the embedded processing unit uses timestamp matching to associate every 3 frames of image data with a set of positioning data, thereby obtaining the spatial coordinates of each shooting location.

[0195] S402. Based on the left view, right view, and spatial coordinates, obtain the defect detection results of the conductor to be detected.

[0196] The defect detection results include indicators of whether there are defective areas in the conductor to be tested, the type of defect in each defective area, and the spatial location of the conductor to be tested.

[0197] In this step, the means of obtaining the indication of whether there is a defect area in the conductor to be detected and the defect type of each defect area based on the left and right views can be referred to the method provided in the foregoing embodiments of this application, which will not be repeated here; in addition, in order to enable the defect detection result to reflect the spatial position of the conductor to be detected, it is also necessary to add the spatial coordinates of the synchronously acquired shooting position to the defect detection result.

[0198] For example, the defect detection results may include the defect type, defect level, and mask image of each defect area identified for the conductor to be detected, as well as the latitude, longitude, and altitude when the conductor to be detected was photographed.

[0199] In one possible implementation, the method provided in this embodiment further includes the following steps a and b:

[0200] Step a: For each defect detection result indicating a defective area in the inspection task, generate alarm information based on the defect detection result of the ground wire to be inspected.

[0201] Specifically, generating alarm information refers to automatically creating notifications or warnings containing detailed defect information based on the conductor / ground wire defect detection results. The purpose is to transform the detected defect information into a format that maintenance personnel can understand and process, providing a basis for subsequent defect handling.

[0202] In one implementation, the alarm information may include the spatial coordinates of the defective line, as well as information such as the defect type and severity level of various defects present. In another implementation, the alarm information may also be a visual report containing a screenshot of the defective area and spatial location markers, along with a brief text description; the information generation process can be automatically triggered based on the defect detection results, filling in relevant data using a preset template.

[0203] Step b: Push alarm information.

[0204] In this step, pushing alarm information refers to proactively sending the generated alarm information to relevant recipients, such as maintenance personnel or the dispatch center, through preset communication channels. The purpose is to ensure that defect information is promptly and accurately communicated to the personnel who need to address it, thereby shortening response time and preventing the fault from escalating.

[0205] The push method may include, but is not limited to, SMS notifications, emails, or pushes through the message interface of a dedicated operation and maintenance platform. This application does not impose specific restrictions on this.

[0206] Optionally, different push channels or priorities can be selected based on the severity of the defects in the line under inspection. This method can guide maintenance personnel to prioritize high-risk defects, rationally allocate human and material resources, and avoid the confusion and omission of key issues caused by pushing all defects at once. Furthermore, it can reduce the ineffective interference of low-risk defects on maintenance personnel, while ensuring that serious defects are addressed promptly, preventing defects from worsening due to delays, thereby reducing line maintenance costs and the probability of safety incidents.

[0207] The methods provided in steps a and b of this implementation, by automatically pushing defects based on the defect detection results of the conductor to be tested, can promptly and proactively convey this key information to relevant maintenance personnel or management systems. This allows maintenance personnel to quickly learn about the defect situation and make rapid decisions and take corresponding measures based on the detailed data such as defect type, location, and severity level contained in the alarm information. For example, they can dispatch maintenance personnel to designated locations for repairs, effectively shortening the cycle from defect discovery to problem resolution, and further improving the maintenance efficiency and safety of conductors and ground wires.

[0208] S403. Based on the defect detection results of each conductor to be inspected at each shooting position in the inspection task, generate a conductor inspection report.

[0209] In this step, a conductor inspection list will be generated based on the defect detection results of each conductor to be inspected in the inspection task.

[0210] The process of generating a conductor / ground wire inspection report involves summarizing, organizing, and formatting all defect detection results obtained during this inspection task. This conductor / ground wire inspection report may include detailed information on each defect of each conductor / ground wire to be inspected during the inspection task.

[0211] In practical applications, report generation can be achieved through automated report generation software. This software can read and parse defect detection result data containing spatial coordinates, and then automatically generate a structured report based on a preset report template.

[0212] The conductor and ground wire defect detection method provided in this application adds the shooting location to the corresponding defect detection report of the line to be inspected, summarizes the defect detection reports of each line to be inspected involved in the inspection task, and generates a conductor and ground wire inspection report accordingly. This allows each detected conductor and ground wire defect to be accurately located in the actual geographical space, improving the convenience of data management and viewing analysis for operation and maintenance personnel, as well as the traceability of line defects and the efficiency of maintenance work.

[0213] Figure 5 This is a schematic diagram of the structure of a defect detection device for a ground wire provided in Embodiment 5 of this application, as shown below. Figure 5 As shown, the defect detection device 50 for ground wires provided in this embodiment includes:

[0214] The first acquisition module 501 is used to acquire the left and right views of the target area synchronously acquired by the binocular imaging device; the target area includes the ground wire to be detected.

[0215] The second acquisition module 502 is used to acquire a first depth map corresponding to the target area based on the left view and the right view; the pixel value of each pixel in the first depth map is the distance between the spatial position of the pixel in the target area and the shooting position.

[0216] The first rejection module 503 is used to reject pixels in the first depth map that are outside a preset distance range to obtain a second depth map;

[0217] The processing module 504 is used to input the left view, the right view, and the second depth map into the pre-trained semantic segmentation model to obtain the image segmentation result; the image segmentation result includes the mask image of each defect region in the ground wire to be detected;

[0218] The third acquisition module 505 is used to acquire the defect detection results of the ground wire to be detected based on the image segmentation results; the defect detection results are used to indicate whether there are defect areas in the ground wire to be detected and the defect type of each defect area.

[0219] The defect detection device 50 for ground wires provided in this embodiment can perform the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0220] Figure 6 This is a schematic diagram of the structure of a defect detection device for a ground wire provided in Embodiment Six of this application, as shown below. Figure 6 As shown, based on the above embodiments, the defect detection device 50 for ground wires provided in this embodiment further includes:

[0221] The second elimination module 506 is used to extract the appearance features of the defect region based on the mask image of the defect region for each defect region.

[0222] Based on apparent characteristics, determine whether the defect area is a pseudo-defect;

[0223] Remove defect areas identified as false defects in the defect detection results;

[0224] Among them, the appearance features include at least one of the following: texture features of defect areas, edge features, area features, and grayscale features.

[0225] The determination module 507 is used to calculate the actual size parameters of each defect region based on the camera intrinsic parameters of the binocular imaging device and the pixel value of each pixel in the second depth map.

[0226] Determine the severity level of the defect area based on the actual size parameters and the preset grading standards;

[0227] Add the defect severity level of each defect area to the defect detection results.

[0228] Generation module 508 is used to obtain the spatial coordinates of the shooting location;

[0229] Add spatial coordinates to the defect detection results;

[0230] Based on the defect detection results of each conductor and ground wire to be inspected at various shooting positions during the inspection task, a conductor and ground wire inspection report is generated.

[0231] The alarm module 509 is used to generate alarm information based on the defect detection results of the ground wire to be tested in each defect detection result indicating a defective area during the inspection task.

[0232] Alarm information is pushed out.

[0233] In one possible implementation, the second acquisition module 502 is specifically used for:

[0234] A semi-global block matching algorithm is used to obtain the disparity maps corresponding to the left and right views;

[0235] Based on the disparity map and the camera intrinsic parameters of the binocular imaging device, calculate the distance between the spatial position of each pixel in the target area and the shooting position in the disparity map;

[0236] Construct a first depth map based on the distance to each pixel.

[0237] In one possible implementation, the semantic segmentation model in the processing module 504 is obtained by pre-training an improved YOLACT real-time instance segmentation model using a sample set based on a loss function; the sample set includes multiple labeled defective conductor samples and normal conductor samples.

[0238] The improved YOLACT real-time instance segmentation model is obtained by modifying at least one pre-defined convolutional layer in the backbone network of the original YOLACT real-time instance segmentation model.

[0239] For each preset convolutional layer, the modified processing is to add a 1×N linear convolutional branch in parallel next to the 3×3 convolutional branch of the preset convolutional layer, and to add a weighted fusion layer based on the attention mechanism in series at the output ends of the 3×3 convolutional branch and the 1×N linear convolutional branch; N is the preset convolutional length; the weighted fusion layer is used to assign weights to the output features of the two branches, and to perform weighted fusion of the output features of the two branches according to the assigned weights.

[0240] The defect detection device 50 for ground wires provided in this embodiment can perform the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0241] Figure 7 A schematic diagram of the processing equipment provided in this application. Figure 7 As shown, the processing device 60 provided in this embodiment includes at least one processor 601 and a memory 602. Optionally, the device 60 further includes a communication component 603. The processor 601, memory 602, and communication component 603 are connected via a bus 604.

[0242] In a specific implementation, at least one processor 601 executes computer execution instructions stored in memory 602, causing at least one processor 601 to perform the above-described method.

[0243] The specific implementation process of processor 601 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0244] For example, the processing device may be a drone or a ground processing terminal.

[0245] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0246] The memory may include read-only memory and random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0247] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0248] This application also provides a computer program product, including a computer program that, when executed, implements the above-described method.

[0249] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed, implement the above-described method.

[0250] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as SRAM, EEPROM, EPROM, PROM, ROM, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0251] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside within an ASIC. Alternatively, the processor and the readable storage medium can exist as discrete components in a device.

[0252] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0253] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0254] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0255] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0256] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0257] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

[0258] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for detecting defects in a ground wire, characterized in that, include: Acquire the left and right views of the target area simultaneously captured by a binocular imaging device; The target area includes the ground wire to be detected; Based on the left view and the right view, a first depth map corresponding to the target area is obtained; the pixel value of each pixel in the first depth map is the distance between the spatial position of the pixel in the target area and the shooting position. Remove pixels in the first depth map that are outside a preset distance range to obtain the second depth map; The left view, the right view, and the second depth map are input into a pre-trained semantic segmentation model to obtain image segmentation results; the image segmentation results include mask images of each defect region in the ground wire to be detected; Based on the image segmentation results, the defect detection results of the ground wire to be detected are obtained; The defect detection results are used to indicate whether there are defective areas in the conductor to be detected and the type of defect in each defective area.

2. The method according to claim 1, characterized in that, The step of obtaining the first depth map corresponding to the target region based on the left view and the right view includes: A semi-global block matching algorithm is used to obtain the disparity maps corresponding to the left view and the right view; Based on the disparity map and the camera intrinsic parameters of the binocular imaging device, calculate the distance between the spatial position of each pixel in the disparity map in the target area and the shooting position; The first depth map is constructed based on the distance corresponding to each pixel.

3. The method according to claim 1, characterized in that, The semantic segmentation model is obtained by pre-training the improved YOLACT real-time instance segmentation model based on a loss function and a sample set; the sample set includes multiple labeled defective conductor samples and normal conductor samples. The improved YOLACT real-time instance segmentation model is obtained by modifying at least one preset convolutional layer in the backbone network of the original YOLACT real-time instance segmentation model. For each preset convolutional layer, the modification process involves adding a 1×N linear convolutional branch in parallel next to the 3×3 convolutional branch of the preset convolutional layer, and adding a weighted fusion layer based on an attention mechanism in series at the output ends of the 3×3 convolutional branch and the 1×N linear convolutional branch; where N is the preset convolutional length; the weighted fusion layer is used to assign weights to the output features of the two branches, and to perform weighted fusion of the output features of the two branches according to the assigned weights.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: For each defect region, the appearance features of the defect region are extracted based on the mask image of the defect region; Based on the apparent characteristics, determine whether the defect region is a pseudo-defect; Remove the defect areas that are identified as false defects in the defect detection results; The apparent features include at least one of the following: texture features of the defect area, edge features, area features, and grayscale features.

5. The method according to claim 4, characterized in that, The method further includes: For each defect region, the actual size parameters of the defect region are calculated based on the camera intrinsic parameters of the binocular imaging device and the pixel value of each pixel in the second depth map. Based on the actual size parameters and the preset grading standards, the severity level of the defect in the defective area is determined; Add the defect severity level of each defect area to the defect detection results.

6. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Obtain the spatial coordinates of the shooting location; Add the spatial coordinates to the defect detection result; Based on the defect detection results of each conductor and ground wire to be inspected at various shooting positions during the inspection task, a conductor and ground wire inspection report is generated.

7. The method according to claim 6, characterized in that, The method further includes: For each defect detection result indicating a defective area in the inspection task, an alarm message is generated based on the defect detection result of the ground wire to be inspected. The alarm information is pushed out.

8. The method according to claim 3, characterized in that, The loss function includes the focusing loss function.

9. A processing device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

Images