Power transmission line detection method, device, equipment, medium and product

Abstract

The embodiment of the invention provides a power transmission line detection method, device and equipment, a medium and a product. The method comprises the following steps: acquiring a power transmission line image through image acquisition equipment arranged on an unmanned aerial vehicle; migrating the style of the binary style image to the power transmission line image through a style migration network and a feature extraction network, and obtaining a fused image; carrying out binarization processing on the fused image to obtain a fused image after binarization; according to the included angle between each edge in the binarized fused image and the flight direction of the unmanned aerial vehicle, determining the edge of which the included angle is greater than a preset angle as a contact edge; determining a plurality of detection areas in the binarized fused image according to the contact edge, the flight direction of the unmanned aerial vehicle and a preset detection window length; and if the black and white pixel proportion in any detection area is smaller than a preset proportion, determining that the power transmission line exists in the detection area. The method is used for improving the power transmission line detection efficiency.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment, medium and product for detecting power lines. Background Art

[0002] With rapid economic growth, power infrastructure construction is experiencing explosive growth. Drone inspection technology, with its efficiency, intelligence, and significant labor cost savings, is becoming a focus of industry research and application, leading the way for more advanced and efficient power inspection models.

[0003] To ensure safe and efficient drone flight, visual methods for drone line inspection primarily rely on identifying transmission line locations to determine flight direction, enabling a bird's-eye view of transmission lines above and along them. Currently, deep learning is the primary method for detecting transmission line locations in transmission line images.

[0004] However, the detection efficiency of deep learning is low. Summary of the Invention

[0005] The embodiments of the present application provide a transmission line detection method, apparatus, equipment, medium and product to achieve the effect of improving detection efficiency.

[0006] In a first aspect, an embodiment of the present application provides a transmission line detection method, comprising:

[0007] The transmission line images are collected by using the image acquisition equipment installed on the UAV;

[0008] Transferring the style of the binary style image to the power line image through a style transfer network and a feature extraction network to obtain a fused image, wherein the binary style image includes alternating black and white stripes, and the direction of the black and white stripes is the flight direction of the UAV;

[0009] performing binarization processing on the fused image to obtain a binarized fused image;

[0010] According to the angle between each edge in the binarized fused image and the flight direction of the UAV, the edge having an angle greater than a preset angle is determined as a contact edge;

[0011] Determining a plurality of detection areas in the binarized fused image according to the contact edge, the flight direction of the UAV, and a preset detection window length;

[0012] If the ratio of black and white pixels in any of the detection areas is less than a preset ratio, it is determined that there is a power transmission line in the detection area.

[0013] In a possible implementation, the first contact edge and the second contact edge both belong to the contact edge, and the first contact edge and the second contact edge both include two edge corners. The multiple detection areas are determined in the binarized fused image based on the contact edge, the flight direction of the drone, and a preset detection window length, including:

[0014] Step a: determining any edge angle of the first contact edge as a starting point;

[0015] Step b: starting from the starting point, extending the preset detection window length along the first contact edge toward another edge angle to generate a detection window;

[0016] Step c, translating the detection window toward the second contact edge along the flight direction of the UAV, and determining the area through which the detection window is translated as the detection area;

[0017] Step d: In the first contact edge, a point at a preset distance from the starting point in the direction of another edge angle is determined as a new starting point;

[0018] Repeat step b to step d until the new starting point is the other edge corner.

[0019] In a possible implementation, translating the detection window toward the second contact edge along the flight direction of the drone, and determining an area through which the detection window is translated as the detection area, includes:

[0020] Step e: translating the detection window along the flight direction of the UAV toward the second contact edge, and determining the area through which the detection window is translated as the initial detection area;

[0021] Step f, calculating a horizontal distance difference and a vertical distance difference based on the first endpoint and the second endpoint of the initial detection area;

[0022] Step g, multiplying the longitudinal distance difference by 2 and subtracting the difference of the lateral distance difference to determine the decision value;

[0023] Step h: determining the coordinate value of the next position and the decision value of the next position according to the decision value and the preset conditions;

[0024] Step i, determining the next position as a new first endpoint;

[0025] Repeat step h and step i until the new first endpoint is the second endpoint, and obtain the detection area.

[0026] In a possible implementation, the preset conditions include:

[0027] If the decision value is greater than or equal to 0, then x k+1 and y k+1 Determine the coordinate value of the next position, and determine the first sum of the decision value and twice the longitudinal distance difference as the decision value of the next position;

[0028] If the decision value is less than 0, then x k+1 and y k Determine the coordinate value of the next position, and determine the second sum of the first sum and twice the horizontal distance difference as the decision value of the next position;

[0029] Wherein, k is a positive integer greater than or equal to 1, and x1 and y1 are the coordinates of the first endpoint.

[0030] In one possible implementation, the style of the binary style image is transferred to the power line image through the style transfer network and the feature extraction network to obtain a fused image, including:

[0031] extracting content features of the power line image through the feature extraction network, and extracting style features of the binary style image through the feature extraction network;

[0032] Extracting image features of the initial image through the style transfer network;

[0033] Calculating a loss function using the content features, the style features, and the image features of the initial image;

[0034] Back-propagating the model parameters of the style transfer network through the loss function to update the initial image;

[0035] When the loss function satisfies a cutoff condition, the updated initial image is determined as the fused image.

[0036] In a possible implementation, binarizing the fused image to obtain a binarized fused image includes:

[0037] Determining a first pixel point in the fused image that is greater than or equal to a preset pixel value, and a second pixel point in the fused image that is less than the preset pixel value;

[0038] The pixel value of the first pixel is set to 255, and the pixel value of the second pixel is set to 0.

[0039] In a second aspect, an embodiment of the present application provides a transmission line detection device, comprising:

[0040] An acquisition module is used to acquire images of power transmission lines using an image acquisition device provided on the UAV;

[0041] a migration module, configured to transfer the style of the binary style image to the power line image through a style migration network and a feature extraction network to obtain a fused image, wherein the binary style image includes alternating black and white stripes, and the direction of the black and white stripes is the flight direction of the UAV;

[0042] a processing module, configured to perform binarization processing on the fused image to obtain a binarized fused image;

[0043] a determination module, configured to determine, based on the angle between each edge in the binarized fused image and the flight direction of the UAV, an edge having an angle greater than a preset angle as a contact edge;

[0044] The determination module is further configured to determine a plurality of detection areas in the binarized fused image according to the contact edge, the flight direction of the UAV, and a preset detection window length;

[0045] The determination module is further configured to determine that there is a power transmission line in any of the detection areas if the ratio of black and white pixels in the detection area is less than a preset ratio.

[0046] In a possible implementation, the first contact edge and the second contact edge both belong to the contact edge, and both the first contact edge and the second contact edge include two edge angles. The determining module is specifically configured to:

[0047] Step a: determining any edge angle of the first contact edge as a starting point;

[0048] Step b: starting from the starting point, extending the preset detection window length along the first contact edge toward another edge angle to generate a detection window;

[0049] Step c, translating the detection window toward the second contact edge along the flight direction of the UAV, and determining the area through which the detection window is translated as the detection area;

[0050] Step d: In the first contact edge, a point at a preset distance from the starting point in the direction of another edge angle is determined as a new starting point;

[0051] Repeat step b to step d until the new starting point is the other edge corner.

[0052] In a possible implementation, the determining module is specifically configured to:

[0053] Step e: translating the detection window along the flight direction of the UAV toward the second contact edge, and determining the area through which the detection window is translated as the initial detection area;

[0054] Step f, calculating a horizontal distance difference and a vertical distance difference based on the first endpoint and the second endpoint of the initial detection area;

[0055] Step g, multiplying the longitudinal distance difference by 2 and subtracting the difference of the lateral distance difference to determine the decision value;

[0056] Step h: determining the coordinate value of the next position and the decision value of the next position according to the decision value and the preset conditions;

[0057] Step i, determining the next position as a new first endpoint;

[0058] Repeat step h and step i until the new first endpoint is the second endpoint, and obtain the detection area.

[0059] In a possible implementation, the preset conditions include:

[0060] If the decision value is greater than or equal to 0, then x k+1 and y k+1 Determine the coordinate value of the next position, and determine the first sum of the decision value and twice the longitudinal distance difference as the decision value of the next position;

[0061] If the decision value is less than 0, then x k+1 and y k Determine the coordinate value of the next position, and determine the second sum of the first sum and twice the horizontal distance difference as the decision value of the next position;

[0062] Wherein, k is a positive integer greater than or equal to 1, and x1 and y1 are the coordinates of the first endpoint.

[0063] In a possible implementation, the migration module is specifically configured to:

[0064] extracting content features of the power line image through the feature extraction network, and extracting style features of the binary style image through the feature extraction network;

[0065] Extracting image features of the initial image through the style transfer network;

[0066] Calculating a loss function using the content features, the style features, and the image features of the initial image;

[0067] Back-propagating the model parameters of the style transfer network through the loss function to update the initial image;

[0068] When the loss function satisfies a cutoff condition, the updated initial image is determined as the fused image.

[0069] In a possible implementation, the processing module is specifically configured to:

[0070] Determining a first pixel point in the fused image that is greater than or equal to a preset pixel value, and a second pixel point in the fused image that is less than the preset pixel value;

[0071] The pixel value of the first pixel is set to 255, and the pixel value of the second pixel is set to 0.

[0072] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor;

[0073] The memory stores computer-executable instructions;

[0074] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0075] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.

[0076] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0077] The power line detection method, apparatus, equipment, medium, and product provided in the embodiments of this application utilize a lightweight style transfer network to highlight the straight line feature areas of the drone's flight direction, thereby improving the problem of lack of prominence of straight line features in power line images due to background objects and color clutter. Subsequently, based on the characteristics of the drone capturing images from a bird's-eye view while flying along the line, the binarized fused image is divided into multiple detection areas with existing widths, and the presence of power lines in each detection area is then detected. Compared to CNNs in the prior art, this application is lightweight and has a fast synthesis speed, which helps improve the efficiency of refined power line detection and better assists drones in achieving autonomous line patrol flights. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0079] Figure 1 Schematic diagram of the process of the transmission line detection method provided in this application Figure 1 ;

[0080] Figure 2 A schematic diagram of a binary style image provided in an embodiment of the present application;

[0081] Figure 3 A schematic diagram of the style transfer process provided in an embodiment of the present application;

[0082] Figure 4 A schematic diagram of a fused image provided in an embodiment of the present application;

[0083] Figure 5 A schematic diagram of the color scale of the fused image and the power line image provided in an embodiment of the present application;

[0084] Figure 6 A schematic diagram of pixel values ​​of a binarized fused image provided in an embodiment of the present application;

[0085] Figure 7 Schematic diagram of the process of the transmission line detection method provided in this application Figure 2 ;

[0086] Figure 8 Schematic diagram of the process of the transmission line detection method provided in this application Figure 3 ;

[0087] Figure 9 Schematic diagram of the process of the transmission line detection method provided in this application Figure 4 ;

[0088] Figure 10 Schematic diagram of the style transfer principle provided for this application;

[0089] Figure 11 A schematic diagram comparing the power transmission line image, fused image, binarized fused image, and processed result image provided in this application;

[0090] Figure 12 A schematic diagram of the structure of the transmission line detection device provided in this application;

[0091] Figure 13 This is a schematic diagram of the structure of the electronic device provided in this application.

[0092] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0093] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0094] First, the application scenarios involved in this application are explained:

[0095] With rapid economic growth, power infrastructure construction has seen explosive growth. Currently, many backbone network facilities responsible for power transformation and transmission are exposed to complex outdoor environments for extended periods. To ensure the continuity and safety of power supply, regular inspections of power infrastructure have become essential. Against this backdrop, drone inspection technology, with its efficiency, intelligence, and significant labor cost savings, is becoming a focus of industry research and application, leading the way towards more advanced and efficient power inspection models.

[0096] To ensure safe and efficient drone flight, visual methods for drone line inspection primarily rely on identifying the location of power lines to determine flight direction, enabling a bird's-eye view of the transmission lines above them. This means that the acquisition of navigational position information often relies on images of the power lines captured by visible light acquisition equipment. While bird's-eye view provides a more comprehensive perspective, visible light acquisition equipment facilitates transmission line inspections. However, this approach is susceptible to interference from the ground background, resulting in lower detection accuracy.

[0097] With the rapid development of deep learning technology, existing techniques have proposed using deep learning to detect power line locations in transmission line images to improve detection accuracy. Specifically, multiple feature extraction modules, such as Feature Pyramid Networks (FPNs) and Region Proposal Networks (RPNs), have been designed within Convolutional Neural Networks (CNNs) to enhance detection performance. The FPN, as a core component, fuses high-level semantic features from deep networks with high-resolution features from shallow networks through a top-down approach and lateral connections, effectively addressing the scale differences of power lines in images caused by varying distances. Building on this feature extraction, the RPN adapts to the slender nature of power lines by presetting anchor boxes with various aspect ratios and employs a non-maximum suppression algorithm to select the most representative candidate regions, significantly improving target localization accuracy. The collaborative work of these modules enables the CNN to accurately identify power lines against complex backgrounds, maintaining stable detection performance even under challenging conditions such as changing lighting, occlusion, and weather interference.

[0098] However, CNN often requires greater device computing power and recognition time, which limits its practical effect and leads to low detection efficiency.

[0099] Based on the above technical problems, the technical concept of this application is as follows:

[0100] The low detection efficiency of existing technologies stems from the intensive computation and large number of parameters involved in convolution operations. When a CNN processes an image, each convolution kernel slides over the input data and performs pixel-by-pixel multiplication and addition. While this local receptive field design effectively extracts features, the computational effort increases exponentially with the number of network layers. Considering that lightweight networks can reduce computational effort and parameter count through a series of optimization strategies while preserving the model's expressive power as much as possible, the detection process can be split into two parts. The first part uses a lightweight network to highlight the power lines in the image acquisition device and generate a fused image. The second part divides the binarized fused image into multiple detection regions based on the relationship between the power lines and the drone's flight direction, and detects the presence of power lines in each detection region. Because the lightweight network in the first part and the detection process in the second part involve less computation, they can accurately improve detection efficiency.

[0101] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0102] Figure 1 Schematic diagram of the process of the transmission line detection method provided in this application Figure 1 ,like Figure 1 As shown, the transmission line detection method can be implemented by the following steps:

[0103] S11. Capture images of power transmission lines using an image acquisition device installed on a drone.

[0104] It should be understood that the image acquisition device is the above-mentioned visible light acquisition device.

[0105] In practical applications, before controlling a drone's flight, the drone's route must be planned to ensure coverage of power lines. Furthermore, the flight altitude, speed, and image acquisition device frequency must be set appropriately to ensure flight safety and the quality of the power line images captured by the image acquisition device. During flight, the drone follows the pre-set route, and the image acquisition device continuously captures images of the power lines at the pre-set frequency. Once captured, the image acquisition device transmits the images to an electronic device via wireless or wired transmission.

[0106] S12. Transfer the style of the binary style image to the power line image through the style transfer network and the feature extraction network to obtain a fused image.

[0107] Among them, since the power lines in the power line image are one or more approximately parallel straight lines, the binary style image can be used as the style source for style transfer to highlight the straight line features of the power line image.

[0108] The binary style image includes alternating black and white stripes, and the direction of the black and white stripes is the flight direction of the drone.

[0109] It should be understood that the flight direction of the UAV can be determined by the direction of the power line at the previous moment. That is, the direction of the power line at the previous moment is the flight direction of the UAV at this moment.

[0110] In practical applications, after a power line image is captured at the current acquisition moment, it is necessary to determine the power line direction determined based on the power line image at the previous acquisition moment as the flight direction of the UAV at the current acquisition moment, and generate a binary style image based on the flight direction of the UAV.

[0111] Specifically, many groups of parallel straight lines covering the entire image can be generated by dividing the image into 8 or 12 equal parts, and the positions between adjacent straight lines are alternately filled with white areas of color level 255 and black areas of color level 0, thereby generating a binary style image.

[0112] Figure 2 This is a schematic diagram of a binary style image provided in the embodiment of this application. Figure 2 As shown, the direction of the transmission line determined according to the transmission line image at the last acquisition moment is horizontal, and a binary style image containing multiple horizontal alternating black and white stripes is generated.

[0113] It should be understood that the horizontal direction is a direction parallel to the bottom edge of the image.

[0114] Figure 3 This is a schematic diagram of the style transfer process provided in the embodiment of the present application. Figure 3 As shown, Figure 3 Part (a) is the power line image, part (b) is the binary style image, and part (c) is the fused image obtained after the style of the binary style image is transferred to the power line image.

[0115] pass Figure 3 It can be seen that after style transfer, there is a big difference between the non-power line area and the power line area in the fused image. Figure 4 A schematic diagram of the fused image provided in the embodiment of the present application is shown as follows: Figure 4 As shown, Figure 4 Parts (a) and (b) are non-transmission line areas, and part (c) is the transmission line area. Since the non-transmission line area does not have the same straight line features (horizontal straight lines) as the binary style image, the non-transmission line area in the fused image appears disorganized; while the transmission line area has the same straight line features as the binary style image, so the transmission line area in the fused image still has straight line features.

[0116] Figure 5 This is a color scale diagram of the fused image and the power line image provided in the embodiment of the present application, as shown in FIG. Figure 5 As shown, Figure 5 (a) shows the color distribution of the transmission line image. Figure 5 (b) shows the color scale distribution of the fused image. It's clear that after style transfer, the color scale of the fused image shifts toward the ends (0 and 255), and the areas with distinct straight line features are concentrated in the ±10 range on either side of the color scale, namely, in the (0, 10) and (245, 255) regions. This feature facilitates subsequent line recognition.

[0117] It should be understood that the specific implementation process and principles of this step will be Figure 9The detailed explanation is given in the illustrated embodiment and will not be repeated here.

[0118] S13. Binarize the fused image to obtain a binarized fused image.

[0119] pass Figure 5 It can be seen that the color levels of the fused image are concentrated in the two areas of (0, 10) and (245, 255). In order to improve the difference between the non-transmission line area and the transmission line area, the fused image can also be binarized to obtain a binarized fused image.

[0120] In one possible implementation, a first pixel in the fused image that is greater than or equal to a preset pixel value and a second pixel in the fused image that is less than the preset pixel value may be determined, and the pixel value of the first pixel may be set to 255, and the pixel value of the second pixel may be set to 0.

[0121] Specifically, the fused image can be binarized using the following formula:

[0122]

[0123] Where th(x,y) is the pixel value of the pixel point (x,y) in the binarized fused image, maxval is the maximum pixel value, which is 255; src(x,y) is the pixel value of the pixel point (x,y) in the fused image, and thresh is the preset pixel value.

[0124] It should be understood that for a common 8-bit image, its grayscale ranges from 0 to 255. The fused image is converted into a binary fused image, where the pixel levels greater than or equal to the preset pixel value are set to one value (usually 255), and the pixel levels less than the preset pixel value are set to another value (usually 0). For the fused image, since the preprocessing in the previous step has already clustered the pixel levels of areas with possible linear features into the two regions (0, 10) and (245, 255), its grayscale image can be directly read and binarized here.

[0125] In the application scenario where a drone flies above and along a power line, and the power line image is taken in daytime visible light, the power line area in the image will tend to be white. After style transfer, it will be concentrated in the 255 color level direction. Therefore, in this scenario, the preset pixel value can be set to 250.

[0126] Figure 6 Schematic diagram of pixel values ​​of the binarized fused image provided in the embodiment of the present application. Figure 6 As shown, the horizontal coordinates of part (a) and part (b) are both pixel points, and the vertical coordinates are both pixel values.

[0127] Part (a) includes the polyline corresponding to the power line image, showing that the pixel values ​​of the power line image are approximately evenly distributed between 0 and 255. Part (a) also includes the polyline corresponding to the preset pixel values, showing that the preset pixel value corresponding to each pixel point is constant.

[0128] Based on part (a), the first pixel whose value is greater than or equal to the preset pixel value is set to 255, and the second pixel whose value is less than the preset pixel value is set to 0. This yields the binarized fused image in part (b). The corresponding polyline of the binarized fused image shows that each pixel in the binarized fused image can only have a value of 0 or 255.

[0129] Optionally, the fused image can also be binarized by methods such as inverse binarization thresholding, truncation thresholding, super-threshold 0 processing, and low-threshold 0 processing to obtain a binarized fused image. The embodiment of the present application does not impose specific restrictions on the binarization processing method.

[0130] S14. According to the angle between each edge in the binarized fused image and the flight direction of the UAV, determine the edge with an angle greater than a preset angle as a contact edge.

[0131] Because power line images contain one or more nearly parallel straight lines that emerge from the edges and continue throughout the entire image, when a drone flies normally along the line, the straight lines are located close to the image's central axis, starting from one edge and ending at the opposite edge. Therefore, based on the angle between each edge in the binarized fused image and the drone's flight direction, the four edges of the binarized fused image can be divided into two categories: contact edges, where the angle between the contact edge and the drone's flight direction is greater than a preset angle; and parallel edges, where the angle between the parallel edge and the drone's flight direction is less than a preset angle.

[0132] For example, the preset angle may be 45°

[0133] S15. Determine multiple detection areas in the binarized fused image according to the contact edge, the flight direction of the UAV, and the preset detection window length.

[0134] In practical applications, considering that the transmission line has a certain width, the length of the detection window can be pre-set according to the width of the transmission line to obtain the preset detection window length. According to the contact edge, the flight direction of the UAV and the preset detection window length, multiple detection areas are determined in the binarized fusion image, so as to subsequently determine whether there is a transmission line in each detection area.

[0135] It should be understood that the implementation process and principle of this step will be Figure 7 The embodiments shown are used for explanation and will not be described in detail here.

[0136] S16. If the ratio of black and white pixels in any detection area is less than a preset ratio, it is determined that there is a power transmission line in the detection area.

[0137] In practical applications, if the detection area is mostly black pixels, that is, most of the pixels have a pixel value of 0, it means that the detection area is mostly background; and if the detection area is mostly white pixels, that is, most of the pixels have a pixel value of 255, it means that the detection area is mostly power lines. Therefore, the black and white pixel ratio in each detection area can be calculated. The black and white pixel ratio is the ratio between black pixels and white pixels in the detection area. If the ratio is less than the preset ratio, it means that the detection area is mostly white pixels, and it is determined that there are power lines in the detection area.

[0138] Optionally, after determining the detection area where the power lines exist, the direction of the power lines may be further determined, thereby laying the foundation for generating a binary style image at the next acquisition moment.

[0139] The power line detection method provided in an embodiment of the present application uses an image acquisition device mounted on a drone to capture a power line image. The style of the binary style image is transferred to the power line image using a style transfer network and a feature extraction network to obtain a fused image. The fused image is binarized to obtain a binarized fused image. Based on the angle between each edge in the binarized fused image and the drone's flight direction, edges with angles greater than a preset angle are identified as contact edges. Based on the contact edges, the drone's flight direction, and a preset detection window length, multiple detection regions are determined in the binarized fused image. If the ratio of black and white pixels in any detection region is less than a preset ratio, a power line is determined to exist in the detection region. The binary style image includes alternating black and white stripes, the direction of which corresponds to the drone's flight direction. In this technical solution, a lightweight style transfer network is used to highlight the linear feature regions of the drone's flight direction, improving the problem of lack of prominence of linear features in power line images due to background objects and color clutter. Then, combined with the characteristics of the image captured by the drone from a bird's-eye view during its flight along the line, the binarized fused image is divided into multiple detection areas with existing widths, and then the presence of power lines in each detection area is detected. Compared with CNN in the prior art, this application has the characteristics of lightweight and fast synthesis speed, which helps to improve the efficiency of refined detection of power lines and better assist drones in realizing autonomous line patrol flights.

[0140] Next, through Figure 7The specific implementation principle and process of the above S15 are explained and will not be repeated here.

[0141] Figure 7 Schematic diagram of the process of the transmission line detection method provided in this application Figure 2 ,like Figure 7 As shown, S15 includes:

[0142] Step a: Determine any edge corner of the first contact edge as the starting point.

[0143] The first contact edge and the subsequent second contact edge both belong to the contact edges shown in S15 , and both the first contact edge and the second contact edge include two edge corners.

[0144] Step b: Starting from the starting point, extend the preset detection window length along the first contact edge toward the other edge angle to generate a detection window.

[0145] Step c: translating the detection window toward the second contact edge along the flight direction of the UAV, and determining the area through which the detection window is translated as the detection area.

[0146] Step d: In the first contact edge, a point that is a preset distance away from the starting point in the direction of another edge angle is determined as a new starting point.

[0147] It should be understood that the preset distance may be less than or equal to the preset detection window length.

[0148] Repeat steps b to d until the new starting point is another edge corner.

[0149] Exemplarily, it is assumed that the left side of the binarized fused image is the first contact edge, and the right side is the second contact edge. The lower edge corner of the left side of the binarized fused image can be determined as the starting point. Extend the preset detection window length from the lower edge corner to the upper edge corner (i.e. upward), and set the line segment between the extended point and the lower edge corner as the detection window. Afterwards, translate the detection window to the right side of the binarized fused image, and determine the area through which the detection window is translated in the binarized fused image as the detection area. Afterwards, when a new detection area needs to be determined, the point at a preset distance upward from the lower edge corner can be determined as the new starting point, and then repeat the above steps until the new starting point is the upper edge corner on the left.

[0150] In the above embodiment, considering that a power line image may contain one or more parallel power lines, that these power lines may intersect the image, and that their directions rarely change, the power line image can be divided into multiple detection areas along the drone's flight direction, with each detection area having a certain width. This avoids the problem of a large number of detection areas resulting from overly narrow divisions, thereby improving detection efficiency.

[0151] It should be understood that in actual applications, in images of power lines captured by a drone in normal flight, there is still a certain offset angle between the power lines and the parallel edges. Therefore, the tolerance for the offset angle can be increased by increasing the length of the preset detection window. Similarly, when dividing the detection area, the detection window can be translated not only along the drone's flight direction toward the second contact edge, but also along the target direction toward the second contact edge. The angle between the target direction and the drone's flight direction is less than the preset deviation angle. In other words, multiple detection areas can be generated for the detection window at the same location.

[0152] Since the smallest unit of the binarized fusion image is a discrete pixel, and the function used to describe the edge of the detection area is continuous, it is inevitable that the edge of the detection area will pass through the middle of the pixel at a small scale. Assume that the edge of the detection area passes through point P k (x k ,y k ), then we need to determine the next point P k+1 What is displayed in pixels (x k+1 ,y k ) or pixels (x k+1 ,y k+1 ) place.

[0153] Figure 8 Schematic diagram of the process of the transmission line detection method provided in this application Figure 3 ,like Figure 8 As shown, the above step c includes:

[0154] Step e: translating the detection window toward the second contact edge along the flight direction of the UAV, and determining the area through which the detection window is translated as the initial detection area.

[0155] Step f: Calculate the horizontal distance difference and the vertical distance difference based on the first endpoint and the second endpoint of the initial detection area.

[0156] Specifically, the absolute value of the abscissa of the first endpoint is subtracted from the abscissa of the second endpoint to determine the horizontal distance difference, and the absolute value of the ordinate of the first endpoint is subtracted from the ordinate of the second endpoint to determine the vertical distance difference.

[0157] Step g: multiply the longitudinal distance difference by 2, and subtract the difference of the transverse distance difference to determine the decision amount.

[0158] Specifically, the decision value can be calculated by the following formula:

[0159] p k =2Δy-Δx

[0160] Wherein, k is a positive integer greater than or equal to 1, Δy is the longitudinal distance difference, and Δx is the lateral distance difference.

[0161] Step h: Determine the coordinate value of the next position and the decision value of the next position based on the decision value and preset conditions.

[0162] If the decision value is greater than or equal to 0, then x k+1 and y k+1 The coordinate value of the next position is determined, and the first sum between the decision amount and 2 times the longitudinal distance difference is determined as the decision amount of the next position.

[0163] That is, p k+1 =p k +2Δy, where p k+1 is the decision value for the next position, x1 and y1 are the coordinates of the first endpoint.

[0164] If the decision value is less than 0, then x k+1 and y k The coordinate value of the next position is determined, and the second sum between the first sum and twice the horizontal distance difference is determined as the decision amount of the next position.

[0165] That is, p k+1 =p k +2Δy-2Δx.

[0166] Step i: Determine the next position as the new first endpoint.

[0167] That is, the coordinate value of the next position is determined as the coordinate value of the new first endpoint, and the decision amount of the next position is determined as the decision amount of the new first endpoint.

[0168] Repeat step h and step i until the new first endpoint becomes the second endpoint, and obtain the detection area.

[0169] Through the above embodiments, the jagged edges of the detection area can be effectively avoided, the edge continuity and the accuracy of the detection area division can be improved, and the accuracy of subsequent transmission line detection can be improved.

[0170] Figure 9 Schematic diagram of the process of the transmission line detection method provided in this application Figure 4 ,like Figure 9 As shown, S12 includes:

[0171] S91. Extract content features of the power line image through a feature extraction network, and extract style features of the binary style image through a feature extraction network.

[0172] Because the feature extraction network extracts different image features through multiple layers of convolution, the network already possesses general image feature extraction capabilities. Therefore, the output of each convolutional layer can be considered a different type of image feature. Convolutional layers closer to the input layer tend to extract features that are more specific to image details, while those closer to the input layer tend to extract features that are more specific to global image features.

[0173] For example, the feature extraction network can be a Visual Geometry Group (VGG)-19 network pre-trained based on ImageNet. It should be understood that the feature extraction network can also be any neural network with image feature extraction capabilities.

[0174] S92. Extract image features of the initial image through the style transfer network.

[0175] Among them, the style transfer network includes multiple convolutional layers, and the features of the initial image are extracted through each convolutional layer.

[0176] S93. Calculate a loss function using the content features, the style features, and the image features of the initial image.

[0177] Among them, the loss function includes style sub-loss, content sub-loss and total variation sub-loss.

[0178] ①Content loss L c

[0179] Content loss L c Used to measure the difference in content between the initial image and the power line image, the square loss function is used:

[0180]

[0181] Where m is the total number of pixels, is the pixel value of pixel i in the initial image determined according to the image features, y i is the pixel value of pixel i in the transmission line image determined according to the content features.

[0182] ② Style loss L s

[0183] Style loss L s It is used to measure the difference in style between the initial image and the power line image. The c×h×w tensor output by the style layer of the feature extraction network can be obtained by the matrix X c×hw To represent the output style features of the style layer, each row vector represents the style sub-feature on a specific channel.

[0184] The style loss subfunction is expressed by the following formula:

[0185]

[0186] g i =XX T

[0187] in, is the pixel value of pixel i in the initial image determined according to the image features, g i is the pixel value of pixel i in the binary style image determined according to the style features.

[0188] ③ Total variable molecular loss L v

[0189] The total variation numerator loss is used to reduce high-frequency noise in the initial image and make the pixel values ​​of adjacent pixels as similar as possible.

[0190] L v =|x i,j -x i+1,j |+|x i,j -x i,j+1 |

[0191] Among them, x i,j are the pixel coordinates in the original image.

[0192] During the entire style transfer process, the weighted sum of the three sub-loss functions is used as the total loss function, and the initial image is updated by adjusting the weight hyperparameters.

[0193] S94. Back-propagate the model parameters of the style transfer network through the loss function to update the initial image.

[0194] The style transfer network’s own parameters are updated through back propagation of the loss function to enhance the image fusion effect, while the feature extraction network does not need to update its parameters.

[0195] S95. When the loss function satisfies a cutoff condition, the updated initial image is determined as the fused image.

[0196] The cutoff condition may be that the loss function value is less than a preset value.

[0197] In the above embodiment, the style transfer network updates the model parameters by calculating the loss function and performing backpropagation, and continuously updates the initial image to make the loss function smaller, so as to ensure that the final updated initial image (fused image) has the style of the binary style image, thereby achieving the purpose of highlighting the straight line features.

[0198] Figure 10 The schematic diagram of the style transfer principle provided for this application is as follows: Figure 10As shown in the figure, the feature extraction network is the VGG-19 network. The feature extraction part of the VGG-19 network uses five VGG blocks, of which the first two blocks each use two 3×3 convolutional layers and one 2×2 maximum pooling layer; the last three blocks each use four 3×3 convolutional layers and one 2×2 maximum pooling layer.

[0199] To preserve local features while capturing as much global information as possible from the power line image, the last convolutional layer of the fourth VGG block is selected as the content layer, and the second convolutional layer of each VGG block is selected as the style layer. In other words, the last convolutional layer of the fourth VGG block in the feature extraction network extracts content features from the power line image, while the second convolutional layer of each VGG block extracts style features from the binary style image.

[0200] Afterwards, the loss function of the style transfer is calculated through forward propagation (in the direction of the solid arrow), and the style transfer model parameters are iterated through back propagation (in the direction of the dotted arrow), that is, the initial image is continuously updated until the loss function meets the cutoff condition and the fused image is obtained.

[0201] Among them, L c is the content loss, L s is the style loss, L v is the total variable molecular loss.

[0202] In practical applications, in order to simplify the operation, the first 30 layers of the VGG-19 network can be determined as the feature extraction network.

[0203] This application sets the power line image that needs to be extracted for straight line features as the content source, and the binary style image with a specific direction (the flight direction of the drone) as the style source. The two are fused through a style transfer network. The fused image can enhance the straight line features in a specific direction and realize style transfer.

[0204] Figure 11 This is a comparison diagram of the power line image, fused image, binarized fused image, and processed result image provided in this application. Figure 11 As shown in the figure, the fused image retains the straight line features of the transmission line in the transmission line image. The binarized fused image highlights the difference between the transmission line area and the non-transmission line area. The yellow area in the processed image is the detection area where the transmission line exists.

[0205] Figure 12 This is a schematic diagram of the structure of the transmission line detection device provided in this application, as shown in Figure 12 As shown, the transmission line detection device 120 provided in this embodiment includes:

[0206] The acquisition module 121 is used to acquire images of power transmission lines using an image acquisition device installed on a UAV.

[0207] The migration module 122 is used to transfer the style of the binary style image to the power line image through the style migration network and the feature extraction network to obtain a fused image. The binary style image includes alternating black and white stripes, and the direction of the black and white stripes is the flight direction of the drone.

[0208] The processing module 123 is configured to perform binarization processing on the fused image to obtain a binarized fused image.

[0209] The determination module 124 is configured to determine, based on the angle between each edge in the binarized fused image and the flight direction of the UAV, edges with angles greater than a preset angle as contact edges.

[0210] The determination module 124 is further configured to determine a plurality of detection areas in the binarized fused image according to the contact edge, the flight direction of the UAV, and a preset detection window length.

[0211] The determination module 124 is further configured to determine that there is a power line in any detection area if the ratio of black and white pixels in the detection area is less than a preset ratio.

[0212] In a possible implementation, the first contact edge and the second contact edge are both contact edges, and both the first contact edge and the second contact edge include two edge angles. The determination module 124 is specifically configured to:

[0213] Step a: Determine any edge corner of the first contact edge as the starting point.

[0214] Step b: Starting from the starting point, extend the preset detection window length along the first contact edge toward the other edge angle to generate a detection window.

[0215] Step c: translating the detection window toward the second contact edge along the flight direction of the UAV, and determining the area through which the detection window is translated as the detection area.

[0216] Step d: In the first contact edge, a point that is a preset distance away from the starting point in the direction of another edge angle is determined as a new starting point.

[0217] Repeat steps b to d until the new starting point is another edge corner.

[0218] In a possible implementation, the determination module 124 is specifically configured to:

[0219] Step e: translating the detection window toward the second contact edge along the flight direction of the UAV, and determining the area through which the detection window is translated as the initial detection area.

[0220] Step f: Calculate the horizontal distance difference and the vertical distance difference based on the first endpoint and the second endpoint of the initial detection area.

[0221] Step g: multiply the longitudinal distance difference by 2, and subtract the difference of the transverse distance difference to determine the decision amount.

[0222] Step h: Determine the coordinate value of the next position and the decision value of the next position based on the decision value and preset conditions.

[0223] Step i: Determine the next position as the new first endpoint.

[0224] Repeat step h and step i until the new first endpoint becomes the second endpoint, and obtain the detection area.

[0225] In a possible implementation, the preset conditions include:

[0226] If the decision value is greater than or equal to 0, then x k+1 and y k+1 The coordinate value of the next position is determined, and the first sum between the decision amount and 2 times the longitudinal distance difference is determined as the decision amount of the next position.

[0227] If the decision value is less than 0, then x k+1 and y k The coordinate value of the next position is determined, and the second sum between the first sum and twice the horizontal distance difference is determined as the decision amount of the next position.

[0228] Wherein, k is a positive integer greater than or equal to 1, and x1 and y1 are the coordinates of the first endpoint.

[0229] In a possible implementation, the migration module 122 is specifically configured to:

[0230] The content features of the power line image are extracted through the feature extraction network, and the style features of the binary style image are extracted through the feature extraction network.

[0231] The image features of the initial image are extracted through the style transfer network.

[0232] A loss function is calculated using content features, the style features, and image features of the initial image.

[0233] The model parameters of the style transfer network are back-propagated through the loss function to update the initial image.

[0234] When the loss function meets the cutoff condition, the updated initial image is determined as the fused image.

[0235] In a possible implementation, the processing module 123 is specifically configured to:

[0236] A first pixel point in the fused image that is greater than or equal to a preset pixel value and a second pixel point in the fused image that is less than the preset pixel value are determined.

[0237] The pixel value of the first pixel is set to 255, and the pixel value of the second pixel is set to 0.

[0238] The transmission line detection provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.

[0239] Figure 13 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 13 As shown, the electronic device 130 provided in this embodiment includes: at least one processor 131 and a memory 132. Optionally, the electronic device 130 also includes a communication component 133. The processor 131, the memory 132 and the communication component 133 are connected via a bus 134.

[0240] During the specific implementation process, at least one processor 131 executes the computer-executable instructions stored in the memory 132, so that the at least one processor 131 performs the above method.

[0241] The specific implementation process of the processor 131 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0242] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules in the processor.

[0243] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory.

[0244] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be classified into address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0245] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0246] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0247] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0248] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in a device as discrete components.

[0249] The division of units is merely a logical functional division; actual implementations may employ alternative divisions, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, any direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units, either through an interface, electrical, mechanical, or other means.

[0250] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0251] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0252] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0253] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0254] Finally, it should be noted that those skilled in the art will readily identify other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The present invention is not limited to the precise structure described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present invention is limited solely by the appended claims.

Claims

1. A transmission line detection method, characterized in that: include: The transmission line images are collected by using the image acquisition equipment installed on the UAV; Transferring the style of the binary style image to the power line image through a style transfer network and a feature extraction network to obtain a fused image, wherein the binary style image includes alternating black and white stripes, and the direction of the black and white stripes is the flight direction of the UAV; performing binarization processing on the fused image to obtain a binarized fused image; According to the angle between each edge in the binarized fused image and the flight direction of the UAV, the edge having an angle greater than a preset angle is determined as a contact edge; Determining a plurality of detection areas in the binarized fused image according to the contact edge, the flight direction of the UAV, and a preset detection window length; If the ratio of black and white pixels in any of the detection areas is less than a preset ratio, it is determined that there is a power transmission line in the detection area.

2. The method according to claim 1, characterized in that The first contact edge and the second contact edge both belong to the contact edge, and the first contact edge and the second contact edge both include two edge corners. According to the contact edge, the flight direction of the drone, and the preset detection window length, multiple detection areas are determined in the binarized fused image, including: Step a: determining any edge angle of the first contact edge as a starting point; Step b: starting from the starting point, extending the preset detection window length along the first contact edge toward another edge angle to generate a detection window; Step c, translating the detection window toward the second contact edge along the flight direction of the UAV, and determining the area through which the detection window is translated as the detection area; Step d: In the first contact edge, a point at a preset distance from the starting point in the direction of another edge angle is determined as a new starting point; Repeat step b to step d until the new starting point is the other edge corner.

3. The method according to claim 2, characterized in that The step of translating the detection window toward the second contact edge along the flight direction of the UAV and determining the area through which the detection window is translated as the detection area includes: Step e: translating the detection window along the flight direction of the UAV toward the second contact edge, and determining the area through which the detection window is translated as the initial detection area; Step f, calculating a horizontal distance difference and a vertical distance difference based on the first endpoint and the second endpoint of the initial detection area; Step g, multiplying the longitudinal distance difference by 2 and subtracting the difference of the lateral distance difference to determine the decision value; Step h: determining the coordinate value of the next position and the decision value of the next position according to the decision value and the preset conditions; Step i, determining the next position as a new first endpoint; Repeat step h and step i until the new first endpoint is the second endpoint, and obtain the detection area.

4. The method according to claim 3, characterized in that The preset conditions include: If the decision value is greater than or equal to 0, then x k+1 and y k+1 Determine the coordinate value of the next position, and determine the first sum of the decision value and twice the longitudinal distance difference as the decision value of the next position; If the decision value is less than 0, then x k+1 and y k Determine the coordinate value of the next position, and determine the second sum of the first sum and twice the horizontal distance difference as the decision value of the next position; Wherein, k is a positive integer greater than or equal to 1, and x1 and y1 are the coordinates of the first endpoint.

5. The method according to any one of claims 1 to 4, characterized in that The style of the binary style image is transferred to the power line image through the style transfer network and the feature extraction network to obtain a fused image, including: extracting content features of the power line image through the feature extraction network, and extracting style features of the binary style image through the feature extraction network; Extracting image features of the initial image through the style transfer network; Calculating a loss function using the content features, the style features, and the image features of the initial image; Back-propagating the model parameters of the style transfer network through the loss function to update the initial image; When the loss function satisfies a cutoff condition, the updated initial image is determined as the fused image.

6. The method according to any one of claims 1 to 4, characterized in that The binarization processing is performed on the fused image to obtain a binarized fused image, comprising: Determining a first pixel point in the fused image that is greater than or equal to a preset pixel value, and a second pixel point in the fused image that is less than the preset pixel value; The pixel value of the first pixel is set to 255, and the pixel value of the second pixel is set to 0.

7. A transmission line detection device, characterized in that: include: An acquisition module is used to acquire images of power transmission lines using an image acquisition device provided on the UAV; a migration module, configured to transfer the style of the binary style image to the power line image through a style migration network and a feature extraction network to obtain a fused image, wherein the binary style image includes alternating black and white stripes, and the direction of the black and white stripes is the flight direction of the UAV; a processing module, configured to perform binarization processing on the fused image to obtain a binarized fused image; a determination module, configured to determine, based on the angle between each edge in the binarized fused image and the flight direction of the UAV, an edge having an angle greater than a preset angle as a contact edge; The determination module is further configured to determine a plurality of detection areas in the binarized fused image according to the contact edge, the flight direction of the UAV, and a preset detection window length; The determination module is further configured to determine that there is a power transmission line in any of the detection areas if the ratio of black and white pixels in the detection area is less than a preset ratio.

8. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 when the computer program is executed by a processor.

Images