Methods, devices, electronic equipment and storage media for inspecting power transmission towers

Abstract

This invention discloses a method, apparatus, electronic device, and storage medium for inspecting transmission towers. The method includes: acquiring inspection images of the transmission towers; inputting the inspection images into a connection detection model to perform connection detection, obtaining a connection detection target image of the transmission tower; inputting the connection detection target image into a ring pin detection model to perform ring pin detection, obtaining a ring pin detection result; and mapping the ring pin detection result back to the inspection image to obtain the inspection result of the transmission tower. In other words, this invention can automatically detect whether there are ring pins in the inspection images using the connection detection model and the ring pin detection model, eliminating the need for manual inspection of the transmission tower inspection images. This improves the speed and accuracy of replacing R-type pins with ring pins in the connection components of the transmission tower, and reduces the time cost of replacing R-type pins with ring pins in the connection components of the transmission tower.

Description

Technical Field

[0001] The embodiments of the present invention relate to target detection technology, and more particularly to a method, apparatus, electronic device and storage medium for detecting ring pins in images of power transmission towers. Background Technology

[0002] To ensure the normal operation of power transmission lines, drones are needed to periodically take pictures of power transmission towers in inspectable areas. Then, the drone inspection images are manually checked to determine whether the connecting parts of which power transmission towers are equipped with pins.

[0003] Currently, the pins used on transmission towers mainly include R-pins and ring pins. R-pins are smaller, and drone footage may capture images of towers where R-pins are not clearly visible due to angle issues, leading to misjudgments about whether R-pins are installed on the connecting parts. Drones, on the other hand, do not capture images of towers where ring pins are not visible due to angle issues, effectively reducing the misjudgment rate when checking for ring pins on the connecting parts. Therefore, manual inspection of transmission tower inspection images is necessary to check for ring pins, and then replacing the R-pins on the connecting parts with ring pins.

[0004] However, manual inspection of transmission towers for ring pins has low accuracy and slow inspection speed, which reduces the speed of replacing R-pins with ring pins in the connecting parts of the transmission towers and increases the time cost of replacing R-pins with ring pins in the connecting parts of the transmission towers. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for inspecting power transmission towers, which can improve the speed and accuracy of replacing R-type pins with ring pins in the connecting parts of power transmission towers, and reduce the time cost of replacing R-type pins with ring pins in the connecting parts of power transmission towers.

[0006] In a first aspect, embodiments of the present invention provide a method for inspecting power transmission towers, the method comprising:

[0007] Acquire inspection images of power transmission towers;

[0008] The inspection image is input into the connection detection model to perform connection detection, thereby obtaining the target image of the connection of the transmission tower.

[0009] The target image of the connection part is input into the ring pin detection model to perform ring pin detection, and the ring pin detection result is obtained.

[0010] The inspection results of the transmission tower are obtained by mapping the detection results of the ring pin back to the inspection image.

[0011] Secondly, embodiments of the present invention provide a transmission tower inspection device, the device comprising:

[0012] The inspection image acquisition module is used to acquire inspection images of transmission towers;

[0013] The target image generation module is used to input the inspection image into the connection detection model to perform connection detection and obtain the target image of the connection of the transmission tower.

[0014] The ring pin detection result generation module is used to input the target image of the connection part into the ring pin detection model to perform ring pin detection and obtain the ring pin detection result.

[0015] The inspection result generation module is used to map the detection result of the ring pin back to the inspection image to obtain the inspection result of the transmission tower.

[0016] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the transmission tower inspection method as described in any of the embodiments of the present invention.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the transmission tower inspection method as described in any of the embodiments of the present invention.

[0018] In this embodiment of the invention, inspection images of transmission towers can be acquired; the inspection images are input into a connection detection model for connection detection to obtain a target image of the transmission tower's connection; the target image of the connection detection is input into a ring pin detection model for ring pin detection to obtain a ring pin detection result; the ring pin detection result is mapped back to the inspection image to obtain the inspection result of the transmission tower. That is, this embodiment of the invention can automatically detect whether there are ring pins in the inspection images using the connection detection model and the ring pin detection model, eliminating the need for manual inspection of the transmission tower's inspection images for ring pins. This solves the problems of low accuracy and slow inspection speed associated with manual inspection of transmission tower inspection images for ring pins, improves the speed and accuracy of replacing R-type pins of the transmission tower's connecting components with ring pins, and reduces the time cost of replacing R-type pins of the transmission tower's connecting components with ring pins. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart of a training method for a connection detection model provided in an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of a connecting portion provided in an embodiment of the present invention;

[0022] Figure 3 This is a flowchart illustrating a training method for a preset ring pin detection model provided in an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of a ring pin provided in an embodiment of the present invention;

[0024] Figure 5 This is a schematic flowchart of a joint training method for a connection detection model and a ring pin detection model provided in an embodiment of the present invention.

[0025] Figure 6 This is a flowchart illustrating the transmission tower inspection method provided in an embodiment of the present invention;

[0026] Figure 7 This is another flowchart illustrating the transmission tower inspection method provided in this embodiment of the invention;

[0027] Figure 8 This is a schematic diagram of the structure of the transmission tower inspection device provided in an embodiment of the present invention;

[0028] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0030] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] Since the transmission tower inspection method provided in this embodiment of the invention requires the use of a connection part detection model and a ring pin detection model, and these models need to undergo corresponding preset connection part detection model training, preset ring pin detection model training, and joint training of the connection part detection model and the ring pin detection model before practical application, the training methods for the connection part detection model, the ring pin detection model, and the joint training methods of the connection part detection model and the ring pin detection model will be further explained before describing the transmission tower inspection method provided in this embodiment of the invention. The specific training processes of the above three stages are as follows:

[0032] Figure 1 This is a flowchart illustrating a training method for a connection detection model provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the training method for the connectivity detection model can specifically include the following steps:

[0033] Step 101: Input the first training image into the preset connection detection model to perform connection detection and obtain detection boxes. The first training image includes the preset connection and the label box of the preset connection.

[0034] The connecting part can be understood as a component in a transmission tower that has a ring pin installed. Figure 2 This is a schematic diagram of a connecting portion provided in an embodiment of the present invention, as shown below. Figure 2 As shown, Figure 2 The connecting part 2 is a component with a ring pin installed; the preset connecting part can be understood as a pre-determined connecting part in the inspection image; the preset connecting part detection model can be understood as the training model of the connecting part detection model; the first training image can be understood as an image in which the position information of the preset connecting part has been manually marked.

[0035] Specifically, the first training image can be input into a preset connection part detection model. After the preset connection part detection model performs preset connection part detection using the object detection algorithm it employs, the preset connection part detection model can output the detection box of the preset connection part. To improve the recall and accuracy of the connection part detection model, the object detection algorithm used by the preset connection part detection model can be a two-stage object detection algorithm with good model accuracy. The two-stage object detection algorithm can be either Faster R-CNN or Cascade R-CNN.

[0036] Step 102: Determine the intersection-union ratio (IUU) of the marker box and the detection box.

[0037] Intersection-over-Union (IoU) can be understood as the overlap ratio between multiple boxes.

[0038] Specifically, the overlap area of ​​the marker box and the detection box can be calculated based on the coordinates of the upper left and lower right corners of the overlapping part. Then, the area of ​​the marker box and the detection box can be calculated based on the coordinates of the marker box and the detection box. Finally, the overlap area, the area of ​​the marker box, and the area of ​​the detection box are substituted into the calculation formula of the intersection-union ratio to obtain the intersection-union ratio of the marker box and the detection box.

[0039] Step 103: Determine the target box from the detection box based on the intersection-union ratio.

[0040] Here, the target box can be understood as the detection box whose intersection-union ratio (IUU) with the marker box exceeds a set IUU value.

[0041] Specifically, by comparing the intersection-over-union (IoU) ratio of the detection box and the marker box with the set IoU value, it can be determined whether the IoU value of the detection box and the marker box exceeds the set IoU value. If the IoU value of the detection box and the marker box exceeds the set IoU value, the detection box whose IoU value exceeds the set IoU value can be identified as the target box.

[0042] For example, if the intersection-over-union ratio (IoU) is set to 0.3 and the IoU of the detection box and the marker box is 0.5, it can be determined that the IoU of the detection box and the marker box exceeds the set IoU value, and the detection box can be identified as the target box.

[0043] Step 104: Determine the loss function of the predicted connection detection model based on the target box and the marker box.

[0044] Step 105: Optimize the model parameters of the preset connection detection model based on the loss function to obtain the connection detection model.

[0045] Specifically, the model parameters of the preset connection detection model can be continuously optimized based on the loss function so that the model parameters of the preset connection detection model are adjusted to the optimal level. Thus, the preset connection detection model with the optimal model parameters can be determined as the connection detection model in the transmission tower inspection method for practical applications.

[0046] Figure 3 This is a flowchart illustrating a training method for a preset ring pin detection model provided in an embodiment of the present invention, as shown below. Figure 3As shown, the training method for the pre-defined ring pin detection model may specifically include the following steps:

[0047] Step 301: Input the second training image into the preset ring pin detection model to perform ring pin detection and obtain the detection box. The second training image includes the preset ring pin located at the preset connection part and the preset ring pin's marker box.

[0048] in, Figure 4 This is a schematic diagram of a ring pin provided in an embodiment of the present invention, as shown below. Figure 4 As shown, Figure 4 The ring pin in the image can be used to pass through the pin hole to prevent the nut from falling off. The preset ring pin can be understood as the ring pin that is predetermined in the inspection image. The preset ring pin detection model can be understood as the training model of the ring pin detection model. The second training image can be understood as the image in which the position information of the preset ring pin has been manually marked.

[0049] Specifically, the second training image can be input into a preset ring pin detection model. After the preset ring pin is detected by the target detection algorithm used by the preset ring pin detection model, the preset ring pin detection model can output the detection box of the preset ring pin. To improve the detection efficiency of the ring pin detection model, the target detection algorithm used by the preset ring pin detection model can be a one-stage target detection algorithm with good model accuracy. The one-stage target detection algorithm can be the YOLOv3 algorithm, and the image feature extraction network in YOLOv3 can adopt the RetinaNet network structure.

[0050] Step 302: Determine the intersection-union ratio of the marker box and the detection box.

[0051] Step 303: Determine the target box from the detection box based on the intersection-union ratio.

[0052] Step 304: Determine the loss function of the preset ring pin detection model based on the target box and the marker box.

[0053] Step 305: Optimize the model parameters of the preset ring pin detection model based on the loss function to obtain the ring pin detection model.

[0054] Specifically, the model parameters of the preset ring pin detection model can be continuously optimized based on the loss function so that the model parameters of the preset ring pin detection model are adjusted to the optimal level. Thus, the preset ring pin detection model with the optimal model parameters can be determined as the connection detection model in the transmission tower inspection method for practical applications.

[0055] Figure 5 This is a flowchart illustrating a joint training method for the connection detection model and the ring pin detection model provided in an embodiment of the present invention, as shown below. Figure 5As shown, the joint training method for the connection detection model and the ring pin detection model can specifically include the following steps:

[0056] Step 501: Input the training image into the preset connection detection model to perform connection detection, and obtain the connection detection training image. The training image includes the preset ring pin located in the preset connection and the marker box of the preset ring pin.

[0057] Step 502: Input the connection part detection training image into the preset ring pin detection model to perform ring pin detection and obtain the detection box.

[0058] Step 503: Determine the intersection-union ratio of the marker box and the detection box.

[0059] Step 504: Determine the target bounding box from the detection box based on the intersection-union ratio.

[0060] Step 505: Determine the loss function of the preset ring pin detection model based on the target box and the marker box, and determine the loss function of the preset connection detection model based on the target box and the marker box.

[0061] Step 506: Optimize the model parameters of the preset ring pin detection model based on the loss function of the preset ring pin detection model to obtain the ring pin detection model; and optimize the model parameters of the preset connection detection model based on the loss function of the preset connection detection model to obtain the connection detection model.

[0062] After completing the above three stages of model training, the trained connection detection model and ring pin detection model can be tested. The test images in the inspection images can be input into the connection detection model and ring pin detection model in sequence to obtain the detection results. The detection results are manually reviewed and fed back to the connection label library and ring pin label library for iterative training. The model accuracy of each detection model is continuously tested. When the model accuracy of each detection model reaches the requirements of practical application, the obtained model can be put into practical application.

[0063] After completing the model training and testing processes, the connection detection model and the ring pin detection model for practical application in the transmission tower inspection method can be obtained. The following describes in detail how the connection detection model and the ring pin detection model are used to execute the transmission tower inspection method provided in this embodiment of the invention. Figure 6 This is a flowchart illustrating a transmission tower inspection method provided in an embodiment of the present invention. This method can be executed by a transmission tower inspection device provided in this embodiment, which can be implemented using software and / or hardware. In a specific embodiment, the device can be integrated into an electronic device. The following embodiments will illustrate this using the integration of the device into an electronic device as an example. (Refer to...) Figure 6 The method may specifically include the following steps:

[0064] Step 601: Obtain inspection images of the transmission towers.

[0065] Among them, inspection images can be understood as images of power transmission towers taken by inspection personnel using inspection photography equipment during inspections.

[0066] Specifically, drones equipped with camera equipment can be used to take inspection images of power transmission towers along the inspection route, and then the inspection images of the power transmission towers stored in the camera equipment can be obtained.

[0067] Step 602: Input the inspection image into the connection detection model to perform connection detection and obtain the target image of the connection of the transmission tower.

[0068] Among them, the target image for detecting the connection part of the transmission tower can be understood as the connection part detection image used for detecting the ring pin.

[0069] Specifically, the inspection image can be input into the connection detection model to perform connection detection, thereby obtaining the connection detection image corresponding to the connection detection model. Then, based on the cross-union ratio and confidence of the connection detection images, the connection detection target image can be selected from the connection detection images.

[0070] For example, the inspection image can be input into the connection detection model to perform connection detection, resulting in three connection detection images corresponding to the model. The cross-union ratios (CUIs) of the three connection detection images are 0.2, 0.4, and 0.5, respectively, and the confidence scores are 0.4, 0.6, and 0.7, respectively. Connection detection images with CUIs greater than 0.3 and confidence scores greater than 0.5 can be selected and identified as the connection detection target images.

[0071] Step 603: Input the target image of the connection part into the ring pin detection model to perform ring pin detection and obtain the ring pin detection result.

[0072] The ring pin detection result can include the position information, confidence information, and feature information of the target ring pin in the connection part detection target image. The target ring pin can be understood as the ring pin in the connection part detection target image. The feature information can be understood as the size information of the target ring pin. The position information can be understood as the coordinate information of the target ring pin in the connection part detection target image within the inspection image.

[0073] For example, if the target ring pin is A, the target image of the connection part detection can be input into the ring pin detection model to perform ring pin detection, and the position information, confidence information and feature information of the target ring pin A in the target image of the connection part detection can be obtained.

[0074] Step 604: Map the ring pin detection results back to the inspection image to obtain the inspection results of the transmission tower.

[0075] The inspection results of transmission towers can be understood as a marked image obtained by mapping the detection results of the ring pins back to the inspection image. The marked image can include the location information, confidence information and feature information of the target ring pin in the detection target image at the connection part.

[0076] Specifically, the location, confidence, and feature information of the target ring pin detected at the connection point can be mapped back to the inspection image. This involves marking the location, confidence, and feature information of the target ring pin in the inspection image, thus obtaining the inspection results for the transmission tower. Inspection personnel can determine the presence of a ring pin in the transmission tower's inspection image by checking whether the inspection results contain these markers.

[0077] In this embodiment of the invention, inspection images of transmission towers can be acquired; the inspection images are input into a connection detection model for connection detection to obtain a target image of the transmission tower's connection; the target image of the connection detection is input into a ring pin detection model for ring pin detection to obtain a ring pin detection result; the ring pin detection result is mapped back to the inspection image to obtain the inspection result of the transmission tower. That is, this embodiment of the invention can automatically detect whether there are ring pins in the inspection images using the connection detection model and the ring pin detection model, eliminating the need for manual inspection of the transmission tower's inspection images for ring pins. This solves the problems of low accuracy and slow inspection speed associated with manual inspection of transmission tower inspection images for ring pins, improves the speed and accuracy of replacing R-type pins of the transmission tower's connecting components with ring pins, and reduces the time cost of replacing R-type pins of the transmission tower's connecting components with ring pins.

[0078] The connection detection model in the transmission tower inspection method provided in this embodiment of the invention may include a first connection detection model and a second connection detection model. The detection algorithm used by the first connection detection model and the detection algorithm used by the second connection detection model may be different. The detection algorithm used by the first connection detection model may be the Faster R-CNN algorithm, and the detection algorithm used by the second connection detection model may be the Cascade R-CNN algorithm. The transmission tower inspection method provided in this embodiment of the invention is further described below, such as... Figure 7 As shown, Figure 7 This is another flowchart illustrating the transmission tower inspection method provided in this embodiment of the invention, which may specifically include the following steps:

[0079] Step 701: Obtain inspection images of the transmission towers.

[0080] Step 702: Input the inspection image into the first connection detection model to perform connection detection, and obtain the first connection detection image corresponding to the first connection detection model.

[0081] Specifically, the inspection image can be input into the first connection detection model to perform connection detection, and first connection detection candidate images can be obtained. Based on the cross-union ratio between the first connection detection candidate images, the first connection detection intermediate image can be determined from the first connection detection candidate images. The first connection detection image can be selected from the first connection detection intermediate image based on the target confidence of the first connection detection intermediate image.

[0082] The target confidence score of the intermediate image detected by the first connection part can be obtained as follows: multiply the original confidence score of the intermediate image detected by the first connection part by the weights of the first connection part detection model to obtain the target confidence score of the intermediate image detected by the first connection part. The original confidence score of the intermediate image detected by the first connection part can be understood as the confidence score of the candidate image detected by the first connection part.

[0083] Furthermore, the inspection image can be input into the first connection detection model to perform connection detection, obtaining first connection detection candidate images. Based on the cross-union ratio (CUI) among the first connection detection candidate images, first connection detection candidate images whose CUI exceeds a set CUI are identified from the first connection detection candidate images. The first connection detection candidate images whose CUI exceeds the set CUI are identified as first connection detection intermediate images. Then, based on the target confidence of the first connection detection intermediate images, first connection detection intermediate images whose target confidence exceeds a set confidence are selected from the first connection detection intermediate images, and the first connection detection intermediate images whose target confidence exceeds the set confidence are identified as first connection detection images.

[0084] For example, if the weight of the first connection detection model is 0.4, the set cross-union ratio (CUP) is 0.3, and the set confidence level is 0.3, the inspection image is input into the first connection detection model for connection detection, resulting in first connection detection candidate images A1, A2, and A3. The CUP between A1 and A2 is 0.3, the CUP between A2 and A3 is 0.5, and the CUP between A3 and A1 is 0.6; the confidence level of A1 is 0.5, the confidence level of A2 is 0.6, and the confidence level of A3 is 0.9. Based on the CUP between each first connection detection candidate image, the first connection detection candidate images whose CUP exceeds the set CUP can be determined from the first connection detection candidate images. The candidate images for connection part detection are A2 and A3. The candidate images for connection part detection whose cross-union ratio (CUI) exceeds the set CUI is determined as the intermediate images for connection part detection, P1 and P2. The original confidence level of P1 is the confidence level of A2, and the original confidence level of P2 is the confidence level of A3. Through the target confidence level calculation method, the target confidence level of P1 is 0.24 and the target confidence level of P2 is 0.36. Then, the intermediate image for connection part detection whose target confidence level exceeds the set confidence level can be selected from P1 and P2 as P2, and the intermediate image for connection part detection whose target confidence level exceeds the set confidence level (P2) is determined as the first connection part detection image S1.

[0085] Step 703: Input the inspection image into the second connection detection model to perform connection detection, and obtain the second connection detection image corresponding to the second connection detection model.

[0086] Specifically, the inspection image is input into the second connection detection model to perform connection detection, and second connection detection candidate images are obtained. Based on the cross-union ratio between the second connection detection candidate images, the second connection detection intermediate images are determined from the second connection detection candidate images. The second connection detection images are then selected from the second connection detection intermediate images based on the target confidence of the second connection detection intermediate images.

[0087] The target confidence of the intermediate image detected by the second connection part can be obtained as follows: multiply the original confidence of the intermediate image detected by the second connection part by the weight of the second connection part detection model to obtain the target confidence of the intermediate image detected by the second connection part.

[0088] It should be noted that the sum of the weights of the first connection detection model and the second connection detection model is 1.

[0089] Furthermore, the inspection image can be input into the second connection detection model to perform connection detection, obtaining candidate images for second connection detection. Based on the cross-union ratio (CUI) among the candidate images, candidate images for second connection detection whose CUI exceeds a set CUI are identified. These candidate images are then designated as intermediate images for second connection detection. Then, based on the target confidence level of these intermediate images, intermediate images for second connection detection whose target confidence exceeds a set confidence level are selected and designated as the final second connection detection images.

[0090] For example, if the weight of the second connection detection model is 0.6, the set cross-union ratio (CUP) is 0.3, and the set confidence level is 0.3, when the inspection image is input into the second connection detection model for connection detection, the resulting candidate images for second connection detection are B1, B2, and B3. The CUP between B1 and B2 is 0.7, the CUP between B2 and B3 is 0.3, and the CUP between B3 and B1 is 0.9; the confidence levels for B1, B2, and B3 are 0.5, 0.6, and 0.8, respectively. Based on the CUP between each candidate image for second connection detection, the images whose CUP exceeds the set CUP can be identified. The candidate images for the second connection part detection are B1 and B3. The candidate images for the second connection part detection whose cross-union ratio (CUI) exceeds the set CUI is determined as the intermediate images for the second connection part detection, P3 and P4. The original confidence of P3 is the confidence of B1, and the original confidence of P4 is the confidence of B3. Through the calculation method of target confidence, the target confidence of P3 is 0.3 and the target confidence of P4 is 0.48. Then, the intermediate image for the second connection part detection whose target confidence exceeds the set confidence is selected from P3 and P4 as P4, and the intermediate image for the second connection part detection whose target confidence exceeds the set confidence (P4) is determined as the second connection part detection image S2.

[0091] Step 704: Determine the target image for connection detection based on the first connection detection image and the second connection detection image.

[0092] Specifically, the first connection detection image and the second connection detection image can be mapped onto the same image to obtain a connection detection mapping image; the connection detection target image is determined from the connection detection mapping image based on the intersection-over-union ratio (IoU) between the various connection detection mapping images. The connection detection mapping image can be understood as an image including the detection boxes of the first and second connection detection images.

[0093] Furthermore, based on the cross-union ratio (CUB) between the various connection part detection mapping images, connection part detection mapping images with a CUB greater than a set CUB can be identified from the connection part detection mapping images, and these connection part detection mapping images with a CUB greater than the set CUB are identified as connection part detection target images.

[0094] For example, if the first connection detection image is S1 and the second connection detection image is S2, and the set cross-union ratio is 0.3, S1 and S2 can be mapped onto the same image to obtain a connection detection mapping image; if the cross-union ratio between S1 and S2 in the connection detection mapping image is 0.5, it can be determined that the cross-union ratio between S1 and S2 exceeds the set cross-union ratio, and S1 and S2 can be identified as connection detection target images.

[0095] Step 705: Input the target image of the connection part detection into the ring pin detection model, so as to extract the ring pin feature image from the target image of the connection part detection using the circular extraction function, and perform ring pin detection on the ring pin feature image using the target detection function to obtain the ring pin detection result.

[0096] Compared to traditional R-type pins, ring pins have a unique circular shape, which is visible even in images taken from different angles using a drone. Therefore, this invention incorporates this circular feature as prior knowledge into the feature extraction function of the feature extraction network, improving its ability to extract circular features. The circular feature extraction function is shown below:

[0097] x = x0 + r cosθ

[0098] y = y0 + r sinθ

[0099] In the above formula, r can represent the radius, θ is a parameter, 0≤θ≤2π, x0 and y0 are the center of the circle, and x and y are the independent variables.

[0100] The ring pin detection model can use the YOLOv3 algorithm, which can include an object detection function and a circular feature extraction function.

[0101] Step 706: Map the position information, confidence information and feature information of the target ring pin in the detection target image at the connection part back to the inspection image to obtain the position information, confidence information and feature information of the target ring pin in the inspection image.

[0102] For example, if the detection result of the ring pin is the location information (L), confidence information (0.8), and feature information (ring pin radius is 2 cm) of the target ring pin A in the detection target image S1 at the connection part, the location information, confidence information, and feature information of the target ring pin A in the detection target image S1 at the connection part can be mapped back to the inspection image to obtain the location information, confidence information, and feature information of the target ring pin in the inspection image. If the inspection personnel can see the markers of the location information, confidence information, and feature information of the target ring pin A in the inspection image of the transmission tower, then it is determined that there is a ring pin in the inspection image of the transmission tower.

[0103] In this embodiment of the invention, the connection detection model and the ring pin detection model can be used to automatically detect whether there are ring pins in the inspection images, eliminating the need for manual inspection of the transmission towers. This solves the problems of low accuracy and slow inspection speed of manual inspection of transmission towers, improves the speed and accuracy of replacing R-type pins of the connecting parts of the transmission towers with ring pins, and reduces the time cost of replacing R-type pins of the connecting parts of the transmission towers with ring pins.

[0104] Figure 8 This is a schematic diagram of a transmission tower inspection device provided in an embodiment of the present invention. This device is suitable for performing the transmission tower inspection method provided in an embodiment of the present invention. Figure 8 As shown, the device may specifically include:

[0105] The inspection image acquisition module 801 is used to acquire inspection images of transmission towers;

[0106] The target image generation module 802 is used to input the inspection image into the connection detection model to perform connection detection and obtain the connection detection target image of the transmission tower.

[0107] The ring pin detection result generation module 803 is used to input the target image of the connection part into the ring pin detection model to perform ring pin detection and obtain the ring pin detection result.

[0108] The inspection result generation module 804 is used to map the ring pin detection result back to the inspection image to obtain the inspection result of the transmission tower.

[0109] Optionally, the connection detection model includes a first connection detection model and a second connection detection model, wherein the detection algorithm used by the first connection detection model is different from the detection algorithm used by the second connection detection model;

[0110] The target image generation module 802 is specifically used for:

[0111] The inspection image is input into the first connection detection model to perform connection detection, and the first connection detection image corresponding to the first connection detection model is obtained.

[0112] The inspection image is input into the second connection detection model to perform connection detection, and the second connection detection image corresponding to the second connection detection model is obtained.

[0113] The connection detection target image is determined based on the first connection detection image and the second connection detection image.

[0114] Optionally,

[0115] The target image generation module 802 inputs the inspection image into the first connection detection model to perform connection detection, and obtains the first connection detection image corresponding to the first connection detection model. This includes: inputting the inspection image into the first connection detection model to perform connection detection, obtaining first connection detection candidate images, determining first connection detection intermediate images from the first connection detection candidate images based on the cross-union ratio between each first connection detection candidate image, and filtering the first connection detection image from the first connection detection intermediate images based on the target confidence of the first connection detection intermediate images.

[0116] The target image generation module 802 inputs the inspection image into the second connection detection model to perform connection detection, and obtains the second connection detection image corresponding to the second connection detection model. This includes: inputting the inspection image into the second connection detection model to perform connection detection, obtaining second connection detection candidate images, determining second connection detection intermediate images from the second connection detection candidate images based on the cross-union ratio between each second connection detection candidate image, and filtering out the second connection detection image from the second connection detection intermediate images according to the target confidence of the second connection detection intermediate images.

[0117] Optionally,

[0118] The target confidence of the first connection part detection intermediate image is obtained by multiplying the original confidence of the first connection part detection intermediate image with the weight of the first connection part detection model to obtain the target confidence of the first connection part detection intermediate image.

[0119] The target confidence of the intermediate image of the second connection part detection is obtained by multiplying the original confidence of the intermediate image of the second connection part detection with the weight of the second connection part detection model to obtain the target confidence of the intermediate image of the second connection part detection.

[0120] Optionally, the target image generation module 802 determines the connection detection target image based on the first connection detection image and the second connection detection image, including:

[0121] The first connection detection image and the second connection detection image are mapped onto the same image to obtain a connection detection mapping image;

[0122] The target image for connection detection is determined from the connection detection mapping images based on the cross-union ratio (CUI) between the various connection detection mapping images.

[0123] Optionally, the annular pin detection model includes a circular extraction function and a target detection function;

[0124] The ring pin detection result generation module 803 is specifically used for:

[0125] The target image of the connection part is input into the ring pin detection model, so as to extract the ring pin feature image from the target image of the connection part using the circular extraction function, and to perform ring pin detection on the ring pin feature image using the target detection function to obtain the ring pin detection result.

[0126] Optionally, the ring pin detection result includes the position information, confidence information, and feature information of the target ring pin in the detection target image of the connection part;

[0127] Inspection result generation module 804 is specifically used for:

[0128] The position information, confidence information, and feature information of the target annular pin in the detection target image at the connection part are mapped back to the inspection image to obtain the position information, confidence information, and feature information of the target annular pin in the inspection image.

[0129] Optionally, the connection detection model and the annular pin detection model are obtained through the following training method:

[0130] The training image is input into the preset connection detection model to perform connection detection, and a connection detection training image is obtained. The training image includes a preset annular pin located at the preset connection and a marker box of the preset annular pin.

[0131] The connection part detection training image is input into a preset ring pin detection model to perform ring pin detection, and a detection box is obtained.

[0132] Determine the intersection-union ratio (IUU) of the marker box and the detection box;

[0133] The target box is determined from the detection box based on the intersection-union ratio;

[0134] The loss function of the preset annular pin detection model is determined based on the target box and the marker box, and the loss function of the preset connection detection model is determined based on the target box and the marker box.

[0135] The model parameters of the preset ring pin detection model are optimized based on the loss function of the preset ring pin detection model to obtain the ring pin detection model; and the model parameters of the preset connection detection model are optimized based on the loss function of the preset connection detection model to obtain the connection detection model.

[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is merely an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0137] The apparatus of this invention can acquire inspection images of transmission towers; input the inspection images into a connection detection model for connection detection to obtain a target image of the transmission tower's connection parts; input the target image of the connection parts for detection into a ring pin detection model for ring pin detection to obtain a ring pin detection result; and map the ring pin detection result back to the inspection image to obtain the inspection result of the transmission tower. In other words, this invention can automatically detect whether there are ring pins in the inspection images using the connection detection model and the ring pin detection model, eliminating the need for manual inspection of the transmission tower's inspection images. This solves the problems of low accuracy and slow inspection speed associated with manual inspection of transmission tower inspection images, improves the speed and accuracy of replacing R-type pins of the transmission tower's connecting parts with ring pins, and reduces the time cost of replacing R-type pins of the transmission tower's connecting parts with ring pins.

[0138] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the transmission tower inspection method provided in any of the above embodiments.

[0139] This invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the transmission tower inspection method provided in any of the above embodiments.

[0140] The following is for reference. Figure 9 It shows a schematic diagram of the structure of a computer system 1000 suitable for implementing an electronic device according to embodiments of the present invention. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0141] like Figure 9 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the computer system 1000. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0142] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1010 as needed so that computer programs read from it can be installed into storage section 1008 as needed.

[0143] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs the functions defined in the system of this invention.

[0144] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0146] The modules and / or units described in the embodiments of the present invention can be implemented in software or hardware. The described modules and / or units can also be housed in a processor; for example, a processor can be described as including an inspection image acquisition module, a target image generation module, a ring pin detection result generation module, and an inspection result generation module. The names of these modules do not necessarily constitute a limitation on the module itself.

[0147] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include:

[0148] Acquire inspection images of transmission towers; input the inspection images into the connection detection model to perform connection detection, and obtain the connection detection target image of the transmission tower; input the connection detection target image into the ring pin detection model to perform ring pin detection, and obtain the ring pin detection result; map the ring pin detection result back to the inspection image to obtain the inspection result of the transmission tower.

[0149] According to the technical solution of the present invention, inspection images of transmission towers can be acquired; the inspection images are input into a connection detection model for connection detection to obtain a target image of the connection of the transmission tower; the target image of the connection detection is input into a ring pin detection model for ring pin detection to obtain a ring pin detection result; the ring pin detection result is mapped back to the inspection image to obtain the inspection result of the transmission tower. That is, the present invention can automatically detect whether there are ring pins in the inspection images using the connection detection model and the ring pin detection model, eliminating the need for manual inspection of the transmission tower inspection images for ring pins. This solves the problems of low accuracy and slow inspection speed of manual inspection of transmission tower inspection images for ring pins, improves the speed and accuracy of replacing R-type pins of the transmission tower connection components with ring pins, and reduces the time cost of replacing R-type pins of the transmission tower connection components with ring pins.

[0150] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for inspecting power transmission towers, characterized in that, The method includes: Acquire inspection images of power transmission towers; The inspection image is input into the connection detection model to perform connection detection, thereby obtaining the target image of the connection of the transmission tower. The target image of the connection part is input into the ring pin detection model to perform ring pin detection, and the ring pin detection result is obtained; wherein, the ring pin detection result includes the position information, confidence information and feature information of the target ring pin in the target image of the connection part; The inspection results of the transmission tower are obtained by mapping the detection results of the ring pin back to the inspection image. The connection detection model and the annular pin detection model are obtained through the following training method: The training image is input into the preset connection detection model to perform connection detection, and a connection detection training image is obtained. The training image includes a preset annular pin located at the preset connection and a marker box of the preset annular pin. The connection part detection training image is input into a preset ring pin detection model to perform ring pin detection, and a detection box is obtained. Determine the intersection-union ratio (IUU) of the marker box and the detection box; The target box is determined from the detection box based on the intersection-union ratio; The loss function of the preset annular pin detection model is determined based on the target box and the marker box, and the loss function of the preset connection detection model is determined based on the target box and the marker box. The model parameters of the preset ring pin detection model are optimized based on the loss function of the preset ring pin detection model to obtain the ring pin detection model; and the model parameters of the preset connection detection model are optimized based on the loss function of the preset connection detection model to obtain the connection detection model. The ring pin detection model includes a circle extraction function and a target detection function. The step of inputting the target image of the connection part into the ring pin detection model to perform ring pin detection and obtain the ring pin detection result includes: The target image of the connection part detection is input into the ring pin detection model to extract the ring pin feature image from the target image of the connection part detection using the circle extraction function, and the ring pin detection is performed on the ring pin feature image using the target detection function to obtain the ring pin detection result; wherein, the circle extraction function is as follows: In the above formula, r represents the radius. For parameters, , , Let x be the center of the circle, and y be the independent variables. The ring pin detection result includes the location information, confidence information, and feature information of the target ring pin in the detection target image at the connection part. Mapping the ring pin detection result back to the inspection image to obtain the inspection result of the transmission tower includes: The position information, confidence information, and feature information of the target annular pin in the detection target image at the connection part are mapped back to the inspection image to obtain the position information, confidence information, and feature information of the target annular pin in the inspection image.

2. The method according to claim 1, characterized in that, The connection detection model includes a first connection detection model and a second connection detection model. The detection algorithm used by the first connection detection model is different from that used by the second connection detection model. The step of inputting the inspection image into the connection detection model to perform connection detection and obtain the target image of the connection of the transmission tower includes: The inspection image is input into the first connection detection model to perform connection detection, and the first connection detection image corresponding to the first connection detection model is obtained. The inspection image is input into the second connection detection model to perform connection detection, and the second connection detection image corresponding to the second connection detection model is obtained. The connection detection target image is determined based on the first connection detection image and the second connection detection image.

3. The method according to claim 2, characterized in that, The step of inputting the inspection image into the first connection detection model to perform connection detection and obtain the first connection detection image corresponding to the first connection detection model includes: inputting the inspection image into the first connection detection model to perform connection detection and obtain the first connection detection candidate image; determining the first connection detection intermediate image from the first connection detection candidate image based on the cross-union ratio between each of the first connection detection candidate images; and filtering the first connection detection image from the first connection detection intermediate image based on the target confidence of the first connection detection intermediate image. The step of inputting the inspection image into the second connection detection model to perform connection detection and obtain the second connection detection image corresponding to the second connection detection model includes: inputting the inspection image into the second connection detection model to perform connection detection and obtain the second connection detection candidate image; determining the second connection detection intermediate image from the second connection detection candidate image based on the cross-union ratio between each of the second connection detection candidate images; and filtering the second connection detection image from the second connection detection intermediate image according to the target confidence of the second connection detection intermediate image.

4. The method according to claim 3, characterized in that, The target confidence of the first connection part detection intermediate image is obtained by multiplying the original confidence of the first connection part detection intermediate image with the weight of the first connection part detection model to obtain the target confidence of the first connection part detection intermediate image. The target confidence of the intermediate image of the second connection part detection is obtained by multiplying the original confidence of the intermediate image of the second connection part detection with the weight of the second connection part detection model to obtain the target confidence of the intermediate image of the second connection part detection.

5. The method according to claim 2, characterized in that, Determining the connection detection target image based on the first connection detection image and the second connection detection image includes: The first connection detection image and the second connection detection image are mapped onto the same image to obtain a connection detection mapping image; The target image for connection detection is determined from the connection detection mapping images based on the cross-union ratio (CUI) between the various connection detection mapping images.

6. A transmission tower inspection device, characterized in that, The device includes: The inspection image acquisition module is used to acquire inspection images of transmission towers; The target image generation module is used to input the inspection image into the connection detection model to perform connection detection and obtain the target image of the connection of the transmission tower. The ring pin detection result generation module is used to input the target image of the connection part detection into the ring pin detection model to perform ring pin detection and obtain the ring pin detection result; wherein, the ring pin detection result includes the position information, confidence information and feature information of the target ring pin in the target image of the connection part detection; The inspection result generation module is used to map the ring pin detection result back to the inspection image to obtain the inspection result of the transmission tower; The connection detection model and the annular pin detection model are obtained through the following training method: The training image is input into the preset connection detection model to perform connection detection, and a connection detection training image is obtained. The training image includes a preset annular pin located at the preset connection and a marker box of the preset annular pin. The connection part detection training image is input into a preset ring pin detection model to perform ring pin detection, and a detection box is obtained. Determine the intersection-union ratio (IUU) of the marker box and the detection box; The target box is determined from the detection box based on the intersection-union ratio; The loss function of the preset annular pin detection model is determined based on the target box and the marker box, and the loss function of the preset connection detection model is determined based on the target box and the marker box. The model parameters of the preset ring pin detection model are optimized based on the loss function of the preset ring pin detection model to obtain the ring pin detection model; and the model parameters of the preset connection detection model are optimized based on the loss function of the preset connection detection model to obtain the connection detection model. The ring pin detection result generation module is specifically used for: The target image of the connection part detection is input into the ring pin detection model to extract the ring pin feature image from the target image of the connection part detection using a circle extraction function, and the ring pin feature image is then used to perform ring pin detection using a target detection function to obtain the ring pin detection result; wherein, the circle extraction function is as follows: In the above formula, r represents the radius. For parameters, , , Let x be the center of the circle, and y be the independent variables. The inspection result generation module is specifically used for: The position information, confidence information, and feature information of the target annular pin in the detection target image at the connection part are mapped back to the inspection image to obtain the position information, confidence information, and feature information of the target annular pin in the inspection image.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the transmission tower inspection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the transmission tower inspection method as described in any one of claims 1 to 5.

Images