Power transmission line hidden danger identification method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a power transmission line hidden danger identification method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: performing target detection on power transmission lines with different sizes in a power transmission line environment image to obtain power transmission line detection frames with different sizes and power transmission line detection frame position information of each power transmission line detection frame; based on the position information of all the power transmission line detection frames, segmenting the plurality of power transmission line detection frames and the background information in the power transmission line environment image to obtain a plurality of power transmission line images containing power transmission lines; for each power transmission line image, updating the power transmission line image into a target power transmission line image consistent with the power transmission line environment image in size, and performing image restoration on the target power transmission line image; and carrying out hidden danger identification on the plurality of updated target power transmission line images to obtain a hidden danger identification result. The method can accurately realize hidden danger identification of the power transmission line.

Description

Technical Field

[0001] The present application relates to the field of power electronics technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for identifying hidden dangers in power transmission lines. Background Art

[0002] With the rapid advancement of global industrialization and urbanization, the stable operation of the power system has become the key to supporting social and economic development. In this context, the safety and reliability of transmission lines, as the core facilities of power transmission, are directly related to the stable power supply and overall operation efficiency of the power grid. Therefore, it is particularly important to identify the potential hidden dangers of transmission lines.

[0003] Traditional methods of identifying hidden dangers in power transmission lines mainly rely on manual visual inspections and ground patrol vehicles. These practices are not only inefficient, but also incapable of dealing with complex and changeable natural environments and dangerous conditions of high-altitude operations. In recent years, in order to improve the safety and efficiency of identifying hidden dangers in power transmission lines, drone technology and machine vision systems have been introduced into the work of identifying hidden dangers in power transmission lines. Specifically, drones are equipped with high-definition cameras and other sensors, which can quickly cover large areas, significantly increase inspection speeds, and reduce the risks of personnel working at high altitudes and in dangerous environments. Machine vision technology automatically analyzes images captured by cameras to assist in identifying problems such as line faults and foreign object entanglement, further improving the efficiency of identifying hidden dangers in power transmission lines.

[0004] However, the current method of identifying hidden dangers in transmission lines is still not accurate enough. Summary of the invention

[0005] Based on this, it is necessary to provide an accurate method, device, computer equipment, computer-readable storage medium and computer program product for identifying hidden dangers in power transmission lines in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for identifying hidden dangers of a power transmission line, comprising:

[0007] Acquire transmission line environment images;

[0008] Performing target detection on transmission lines of different sizes in the transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame;

[0009] Based on the position information of all the transmission line detection frames, multiple transmission line detection frames and background information in the transmission line environment image are segmented to obtain multiple transmission line images containing the transmission lines;

[0010] For each power transmission line image, the power transmission line image is updated to a target power transmission line image having a size consistent with the power transmission line environment image, and the target power transmission line image is restored to obtain an updated target power transmission line image;

[0011] Hidden danger identification is performed on multiple updated target transmission line images to obtain hidden danger identification results.

[0012] In one of the embodiments, target detection is performed on transmission lines of different sizes in a transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame, including:

[0013] Through the preloaded visual detection model, target detection is performed on transmission lines of different sizes in the transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame.

[0014] In one embodiment, the preloaded visual detection model includes a base network, a feature pyramid network, and a classification and regression subnetwork;

[0015] Through the preloaded visual detection model, target detection is performed on the transmission lines of different sizes in the transmission line environment image, and the transmission line detection frames of different sizes in the transmission line environment image and the transmission line detection frame position information of each transmission line detection frame are obtained, including:

[0016] The basic network is used to extract features of the power transmission line environment image at different scales to obtain multiple power transmission line feature images;

[0017] Performing feature fusion processing on multiple power transmission line feature images through a feature pyramid network to obtain at least one first fused feature image;

[0018] Target detection is performed on each first fused feature image through a classification regression subnetwork, a plurality of first object detection frames of different sizes are generated on each first fused feature image, and the first category and the first object detection frame position information of each first object detection frame are detected, and the first object detection frame whose first category is a transmission line is used as a transmission line detection frame, and the first object detection frame position information of the transmission line detection frame is used as the transmission line detection frame position information of the transmission line detection frame.

[0019] In one embodiment, hidden danger identification is performed on a plurality of updated target transmission line images to obtain hidden danger identification results, including:

[0020] For each updated target transmission line image, a preloaded image recognition model is used to identify hidden dangers in the updated target transmission line image, and the hidden danger category and hidden danger location information of each hidden danger in the updated target transmission line image are obtained;

[0021] Based on the hidden danger category and hidden danger location information of each hidden danger in all updated target transmission line images, a hidden danger identification result is generated.

[0022] In one embodiment, the image recognition model includes an input layer, a skeleton network, a neck network, and a head network;

[0023] Through the preloaded image recognition model, the updated target transmission line image is used to identify hidden dangers, and the hidden danger category and hidden danger location information of each hidden danger in the updated target transmission line image are obtained, including:

[0024] Outputting the updated target power transmission line image to the skeleton network through the input layer;

[0025] extracting at least one target image feature having an importance higher than a preset importance threshold in the updated target power transmission line image through the skeleton network;

[0026] Performing feature enhancement on at least one target image feature through a neck network, and performing feature fusion on the target image feature with the target image feature after the feature enhancement to obtain a second fused feature image;

[0027] The second fused feature image is subjected to target detection through the head network, a second object detection frame of each hidden danger in the second fused feature image is generated, and the hidden danger category and hidden danger position information of each second object detection frame are detected.

[0028] In one embodiment, after performing hidden danger identification on a plurality of updated target transmission line images to obtain hidden danger identification results, the method further includes:

[0029] When the hidden danger identification result indicates that hidden dangers exist, hidden danger categories and hidden danger location information of all hidden dangers are obtained from the hidden danger identification result, and hidden danger prompt messages are pushed according to the hidden danger categories and hidden danger location information of all hidden dangers;

[0030] When the hidden danger identification result indicates that there is no hidden danger, the normal transmission line message is pushed.

[0031] In a second aspect, the present application also provides a transmission line hidden danger identification device, comprising:

[0032] An image acquisition module, used to acquire an image of the power transmission line environment;

[0033] The target detection module is used to perform target detection on the transmission lines of different sizes in the transmission line environment image, and obtain the transmission line detection frames of different sizes in the transmission line environment image and the transmission line detection frame position information of each transmission line detection frame;

[0034] An image segmentation module is used to segment multiple transmission line detection frames and background information in the transmission line environment image based on the position information of all transmission line detection frames to obtain multiple transmission line images containing transmission lines;

[0035] A size conversion module is used to update each power transmission line image to a target power transmission line image having the same size as the power transmission line environment image, and perform image restoration on the target power transmission line image to obtain an updated target power transmission line image;

[0036] The hidden danger identification module is used to identify hidden dangers on multiple updated target transmission line images to obtain hidden danger identification results.

[0037] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0038] Acquire transmission line environment images;

[0039] Performing target detection on transmission lines of different sizes in the transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame;

[0040] Based on the position information of all the transmission line detection frames, multiple transmission line detection frames and background information in the transmission line environment image are segmented to obtain multiple transmission line images containing the transmission lines;

[0041] For each power transmission line image, the power transmission line image is updated to a target power transmission line image having a size consistent with the power transmission line environment image, and the target power transmission line image is restored to obtain an updated target power transmission line image;

[0042] Hidden danger identification is performed on multiple updated target transmission line images to obtain hidden danger identification results.

[0043] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0044] Acquire transmission line environment images;

[0045] Performing target detection on transmission lines of different sizes in the transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame;

[0046] Based on the position information of all the transmission line detection frames, multiple transmission line detection frames and background information in the transmission line environment image are segmented to obtain multiple transmission line images containing the transmission lines;

[0047] For each power transmission line image, the power transmission line image is updated to a target power transmission line image having a size consistent with the power transmission line environment image, and the target power transmission line image is restored to obtain an updated target power transmission line image;

[0048] Hidden danger identification is performed on multiple updated target transmission line images to obtain hidden danger identification results.

[0049] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0050] Acquire transmission line environment images;

[0051] Performing target detection on transmission lines of different sizes in the transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame;

[0052] Based on the position information of all the transmission line detection frames, multiple transmission line detection frames and background information in the transmission line environment image are segmented to obtain multiple transmission line images containing the transmission lines;

[0053] For each power transmission line image, the power transmission line image is updated to a target power transmission line image having a size consistent with the power transmission line environment image, and the target power transmission line image is restored to obtain an updated target power transmission line image;

[0054] Hidden danger identification is performed on multiple updated target transmission line images to obtain hidden danger identification results.

[0055] The above-mentioned power transmission line hidden danger identification method, device, computer equipment, computer-readable storage medium and computer program product firstly perform target detection on the power transmission line environment image, and not only locate the large-sized power transmission lines in the power transmission line environment image, but also locate the small-sized power transmission lines, obtain multiple power transmission line detection frames and the power transmission line detection frame position information of each power transmission line detection frame, and then segment the multiple power transmission line detection frames and the background information in the power transmission line environment image based on the position information of all power transmission line detection frames, and obtain multiple power transmission line maps containing power transmission lines but not containing background information. Then, multiple transmission line images are updated to target transmission line images that are consistent with the size of the transmission line environment image, so that the transmission line images are subsequently easy to detect and identify, and image restoration is performed on each target transmission line image to obtain multiple updated target transmission line images, so that the updated target transmission line images are clearer. Finally, since the updated target transmission line images at this time are clear images that do not contain background information and whose sizes are easy to detect and identify, hidden dangers can be identified for multiple updated target transmission line images to obtain accurate hidden danger identification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 A diagram showing an application environment of a method for identifying hidden dangers of a power transmission line in one embodiment;

[0058] Figure 2 A schematic diagram of a flow chart of a method for identifying hidden dangers of a power transmission line in one embodiment;

[0059] Figure 3 A schematic diagram of a flow chart of a method for identifying hidden dangers of a power transmission line in another embodiment;

[0060] Figure 4 It is a structural block diagram of a device for identifying hidden dangers of a power transmission line in one embodiment;

[0061] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are used to explain the present application and are not used to limit the present application.

[0063] The method for identifying hidden dangers of power transmission lines provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 and the shooting device 106 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The shooting device 106 can be placed in a fixed position or carried on a drone.

[0064] The user triggers the power transmission line hidden danger identification control on the terminal 102, and the terminal 102 responds to the trigger operation and controls the camera 106 placed at a fixed position to start up to shoot the power transmission line environment image of the power transmission line, or controls the drone to take off according to a preset path, and controls the camera 106 carried by the drone to start up to shoot the power transmission line environment image of the power transmission line. The camera 106 transmits the captured power transmission line environment image back to the terminal 102, and the terminal 102 uploads the power transmission line environment image to the server 104, so as to identify the power transmission line hidden danger based on the power transmission line environment image.

[0065] The server 104 obtains a power transmission line environment image; performs target detection on the power transmission line environment image to obtain a power transmission line detection frame in the power transmission line environment image and power transmission line detection frame position information of the power transmission line detection frame; based on the power transmission line detection frame position information, segments the power transmission line detection frame and background information in the power transmission line environment image to obtain a power transmission line image including the power transmission line; updates the power transmission line image to a target power transmission line image having the same size as the power transmission line environment image, performs image restoration on the target power transmission line image, and updates the target power transmission line image; performs hidden danger identification on the updated target power transmission line image to obtain a hidden danger identification result.

[0066] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0067] In an exemplary embodiment, Figure 2 As shown, a method for identifying hidden dangers in a transmission line is provided. Figure 1 The server 104 in FIG. 1 is used as an example for explanation. Among them:

[0068] S100, acquiring a transmission line environment image.

[0069] The power transmission line environment image is an image including the power transmission line and the environment in which the power transmission line is located.

[0070] Specifically, the user triggers the power transmission line hidden danger identification control on the terminal, and the terminal responds to the trigger operation, controls the camera placed at a fixed position to start, so as to take the power transmission line environment image of the power transmission line, or controls the drone to take off according to a preset path, and controls the camera on the drone to start, so as to take the power transmission line environment image of the power transmission line. The camera transmits the captured power transmission line environment image back to the terminal, and the terminal uploads the power transmission line environment image to the server, so as to identify the power transmission line hidden danger based on the power transmission line environment image.

[0071] In some embodiments, the transmission line environment image may also be captured in advance by a shooting device and stored locally in the terminal. When the user triggers the transmission line hidden danger identification control on the terminal, the terminal responds to the trigger operation and obtains the transmission line environment image required to be identified locally to perform transmission line hidden danger identification.

[0072] S200, performing target detection on transmission lines of different sizes in the transmission line environment image, and obtaining transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame.

[0073] Among them, the task of target detection is to find all targets (objects) of interest in the image and determine their categories and locations.

[0074] Specifically, target detection is performed on the transmission line environment image, a transmission line detection frame corresponding to the transmission line is found from the transmission line environment image, and the transmission line detection frame position information of the transmission line detection frame is detected. Furthermore, in the related art, in complex scenes, it is generally easier to identify wide lines, while it is more difficult to identify slender transmission lines. However, the present application can accurately detect targets of transmission lines of different sizes in the transmission line environment image, that is, it can not only identify wide transmission lines, but also slender transmission lines.

[0075] That is, target detection is performed on transmission lines of different sizes in the transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame. In practical applications, the method of performing target detection on the transmission line environment image can be achieved by means of deep learning.

[0076] In an exemplary embodiment, the power transmission line detection frame position information of each power transmission line detection frame is generally upper left corner coordinate information and lower right corner coordinate information of the power transmission line detection frame.

[0077] S300, based on the position information of all the transmission line detection frames, segment the multiple transmission line detection frames and the background information in the transmission line environment image to obtain multiple transmission line images containing the transmission lines.

[0078] Specifically, based on the position information of all transmission line detection frames, multiple transmission line detection frames and background information in the transmission line environment image are segmented, that is, the transmission line environment image is cropped based on the position information of each transmission line detection frame to obtain an image that does not contain background information but simply contains the transmission line detection frame, and then multiple transmission line images containing transmission lines are obtained.

[0079] S400 , for each power transmission line image, updating the power transmission line image to a target power transmission line image having the same size as the power transmission line environment image, and performing image restoration on the target power transmission line image to obtain an updated target power transmission line image.

[0080] Specifically, since the size of some transmission line images obtained after segmentation with the background information may be relatively small and difficult to detect and identify, the present application will also perform image enlargement processing on each transmission line image, and update the transmission line image to a target transmission line image that is consistent with the size of the transmission line environment image, and the target transmission line image is still a transmission line image that does not contain background information.

[0081] Furthermore, the transmission line image after segmentation from the background information is often a relatively small and blurred image. Therefore, it is necessary to restore the target transmission line image through image restoration means, update the target transmission line image, and obtain an image that is convenient for subsequent hidden danger identification. In practical applications, image restoration means generally include image restoration means such as defogging and denoising, in addition to image interpolation and other means.

[0082] S500, performing hidden danger identification on a plurality of updated target transmission line images to obtain hidden danger identification results.

[0083] Among them, the hidden dangers are essentially the broken strands, scattered strands, entanglement of foreign objects and other hidden dangers that may exist on the transmission lines. The cause of the broken strands is lightning strike, which affects the safety of the line and threatens the reliability of communications.

[0084] Specifically, hidden dangers are identified for multiple updated target transmission line images to obtain hidden danger identification results. Generally, the hidden danger identification method can also be achieved by means of deep learning. Further, the hidden danger identification results include the category of hidden dangers and the location information of hidden dangers.

[0085] In the above-mentioned transmission line hidden danger identification method, first, target detection is performed on the transmission line environment image, which can not only locate large-sized transmission lines in the transmission line environment image, but also locate small-sized transmission lines, and obtain multiple transmission line detection frames and transmission line detection frame position information of each transmission line detection frame, and then based on all transmission line detection frame position information, multiple transmission line detection frames and background information in the transmission line environment image are segmented to obtain multiple transmission line images containing transmission lines but not containing background information, and then, the multiple transmission line images are updated to target transmission line images with the same size as the transmission line environment image, so that the transmission line images are easy to detect and identify later, and each target transmission line image is restored to obtain multiple updated target transmission line images, and the updated target transmission line images will be clearer. Finally, since the updated target transmission line image at this time is a clear image that does not contain background information and has a size that is easy to detect and identify, the multiple updated target transmission line images can be used for hidden danger identification to obtain accurate hidden danger identification results.

[0086] In an exemplary embodiment, target detection is performed on transmission lines of different sizes in a transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame, wherein:

[0087] Through the preloaded visual detection model, target detection is performed on transmission lines of different sizes in the transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame.

[0088] Specifically, in order to realize target detection of transmission lines of different sizes in the transmission line environment image, the present application provides a visual detection model, such as the RetinaNet model. A slender object adaptive module is added to the RetinaNet model, which can not only detect large-sized transmission lines, but also small-sized transmission lines in the transmission line environment image.

[0089] Among them, the RetinaNet model is a single-stage target detection model with high accuracy and robustness when detecting small targets in a large background. The single-stage target detection model refers to directly performing target classification and bounding box regression from the feature map, without the need to first generate candidate regions like the two-stage method, which greatly speeds up the detection speed.

[0090] In this embodiment, by adding a visual detection model of a slender object adaptive module, it is possible to accurately detect transmission lines of different sizes in a transmission line environment image.

[0091] In an exemplary embodiment, the preloaded visual detection model includes a base network, a feature pyramid network, and a classification regression subnetwork;

[0092] Through the preloaded visual detection model, target detection is performed on the transmission lines of different sizes in the transmission line environment image, and the transmission line detection frames of different sizes in the transmission line environment image and the transmission line detection frame position information of each transmission line detection frame are obtained, including:

[0093] The basic network is used to extract features of the power transmission line environment image at different scales to obtain multiple power transmission line feature images; the feature pyramid network is used to perform feature fusion processing on the multiple power transmission line feature images to obtain at least one first fused feature image; the classification and regression subnetwork is used to perform target detection on each first fused feature image to generate multiple first object detection frames of different sizes on each first fused feature image, and the first category and the first object detection frame position information of each first object detection frame are detected, and the first object detection frame whose first category is the power transmission line is used as the power transmission line detection frame, and the first object detection frame position information of the power transmission line detection frame is used as the power transmission line detection frame position information of the power transmission line detection frame.

[0094] Specifically, the architecture of the preloaded visual detection model, for example, the RetinaNet model, mainly includes three network structures: the basic network, the feature pyramid network, and the classification regression sub-network. The functions of each part are as follows:

[0095] The basic network is used to extract features of different scales from the transmission line environment image to generate transmission line feature images of different scales. These transmission line feature images will be used for subsequent target classification tasks and position regression tasks.

[0096] Feature pyramid network,The feature pyramid network is built on top of the basic network, and uses top-down paths and lateral connections to enhance the representation of high-level semantic information on feature maps of all scales, realizes feature fusion processing of multiple transmission line feature images, obtains at least one first fused feature image, and performs target detection on each first fused feature image, generating multiple first object detection boxes of different sizes on each first fused feature image.

[0097] More specifically, there are feature images with high semantic level but low resolution, and feature images with low semantic level but high resolution in multiple transmission line feature images. The feature pyramid network uses feature images with high semantic level but low resolution as deep feature images, and uses feature images with low semantic level but high resolution as shallow feature images. First, the deep feature map of the 5th layer is moved up step by step through upsampling (such as bilinear interpolation or nearest neighbor interpolation) to make its size match the feature image of the next layer, that is, the 4th layer, and then added to the feature image of the 4th layer through lateral connection to obtain a feature fused feature image, and then the feature fused feature image is further moved up through upsampling to match the feature image of the 2nd layer, and then added to the feature image of the 3rd layer through lateral connection to obtain a new feature fused feature image, and the above steps are repeated to perform feature fusion step by step in the feature pyramid until all required scales are covered. Among them, taking the addition of the feature image of the 4th layer through the lateral connection as an example, a lateral connection is added between each upsampled deep feature map and its corresponding shallow feature map. The lateral connection means that the number of channels is first adjusted by 1x1 convolution to ensure that the 5th layer has the same number of channels as the upsampled layer, and then added to the feature image of the 4th layer to obtain a fused feature image, thereby combining deep semantic information with shallow spatial detail information. By this means, the feature pyramid network can effectively process objects of different sizes and make predictions at multiple scales to obtain at least one first fused feature image.

[0098] Finally, a classification regression subnetwork is set at each level of the feature pyramid, and targets of different sizes can be detected at each level, thereby improving the accuracy of detecting slender objects. That is, at each level, the first fused feature image in the corresponding level is subjected to target detection, and multiple first object detection frames of different sizes are generated on each first fused feature image. Further, the classification regression subnetwork includes a classification subnetwork and a regression subnetwork, the classification subnetwork is responsible for predicting the category of the first object detection frame, and the regression subnetwork is responsible for predicting the offset of each first object detection frame relative to its corresponding real boundary frame. Therefore, the first category of each first object detection frame can be detected by the classification subnetwork, and the first object detection frame position information of each first object detection frame can be detected by the regression subnetwork. Then, the first object detection frame of the first category of the transmission line is used as the transmission line detection frame of the transmission line in the transmission line environment image, and the first object detection frame position information of the transmission line detection frame is used as the transmission line detection frame position information of the transmission line detection frame. In practical applications, the transmission line detection frame position information is the upper left corner coordinate information and the lower right corner coordinate information of the transmission line detection frame.

[0099] In the above embodiment, by setting the basic network, feature pyramid network and classification regression sub-network in the preloaded visual detection model, the visual detection model can accurately detect targets of transmission lines of various sizes, and can not only identify wide transmission lines, but also identify slender transmission lines more accurately.

[0100] In an exemplary embodiment, hidden danger identification is performed on a plurality of updated target transmission line images to obtain hidden danger identification results, including:

[0101] For each updated target transmission line image, hidden dangers are identified on the updated target transmission line image through a preloaded image recognition model to obtain the hidden danger category and hidden danger location information of each hidden danger in the updated target transmission line image; based on the hidden danger category and hidden danger location information of each hidden danger in all updated target transmission line images, a hidden danger identification result is generated.

[0102] Specifically, each updated target transmission line image contains an image of a transmission line, and hidden dangers need to be identified for each updated target transmission line image corresponding to the transmission line. The means for identifying hidden dangers for any updated target transmission line image include:

[0103] Through the preloaded image recognition model, hidden dangers are identified for the updated target transmission line image, and it is determined whether there are hidden dangers such as broken strands, foreign objects entangled, etc. on the transmission line in the updated target transmission line image. If hidden dangers exist, the hidden danger category and hidden danger location information of each hidden danger can also be obtained. Finally, based on the hidden danger category and hidden danger location information of each hidden danger in all updated target transmission line images, the hidden danger identification result is generated. In other words, the hidden danger identification result refers to the hidden danger category and hidden danger location information of all hidden dangers of all transmission lines in the transmission line environment image. In practical applications, the preloaded image recognition model is generally the Yolov5 model.

[0104] In the above embodiment, hidden dangers are identified on the updated target transmission line image through a preloaded image recognition model to obtain the hidden danger category and hidden danger location information of each hidden danger in the updated target transmission line image, and then the hidden danger category and hidden danger location information of each hidden danger in all updated target transmission line images are collected to accurately obtain the hidden danger identification result of the final transmission line environment image.

[0105] In an exemplary embodiment, the image recognition model includes an input layer, a skeleton network, a neck network, and a head network;

[0106] Through the preloaded image recognition model, the updated target transmission line image is used to identify hidden dangers, and the hidden danger category and hidden danger location information of each hidden danger in the updated target transmission line image are obtained, including:

[0107] The updated target transmission line image is output to the skeleton network through the input layer; at least one target image feature whose importance is higher than a preset importance threshold in the updated target transmission line image is extracted through the skeleton network; at least one target image feature is feature enhanced through the neck network, and the target image feature is feature fused with the target image feature after feature enhancement to obtain a second fused feature image; target detection is performed on the second fused feature image through the head network, a second object detection frame of each hidden danger in the second fused feature image is generated, and the hidden danger category and hidden danger location information of each second object detection frame are detected.

[0108] Specifically, the image recognition model, for example, the Yolov5 model includes an input layer, a skeleton network, a neck network, and a head network. The functions of each part are as follows:

[0109] The input layer is used to support image input of various sizes. For example, it can obtain an updated target transmission line image and output the updated target transmission line image to the skeleton network; the skeleton network is used to extract high-level features from the input image, that is, to extract at least one target image feature whose importance is higher than a preset importance threshold from the updated target transmission line image; the neck network is used to perform feature enhancement on each target image feature and fuse features of different scales to obtain a second fused feature image; the head network is used for final target classification and bounding box regression, that is, to perform target detection on the second fused feature image, generate a second object detection frame for each hidden danger in the second fused feature image, and detect the hidden danger category and hidden danger location information of each second object detection frame. The hidden danger category includes but is not limited to hidden dangers such as broken strands, foreign body entanglement, etc., so as to identify different types of hidden dangers that may exist on the transmission line.

[0110] In the above embodiment, hidden dangers are identified on the updated target transmission line images through the image recognition model, and the hidden danger category and hidden danger location information of each hidden danger in all updated target transmission line images can be accurately obtained to achieve accurate hidden danger identification.

[0111] In an exemplary embodiment, Figure 3 As shown, after S500, the method further includes:

[0112] S620, when the hidden danger identification result indicates that hidden dangers exist, hidden danger categories and hidden danger location information of all hidden dangers are obtained from the hidden danger identification result, and hidden danger prompt messages are pushed according to the hidden danger categories and hidden danger location information of all hidden dangers.

[0113] S640: When the hidden danger identification result indicates that there is no hidden danger, a message that the power transmission line is normal is pushed.

[0114] Specifically, the hidden danger identification result represents the existence of the hidden danger or the non-existence of the hidden danger. When the hidden danger identification result represents the existence of the hidden danger, the hidden danger identification result also includes the hidden danger category and hidden danger location information of each hidden danger in all transmission lines.

[0115] When the hidden danger identification results indicate the existence of hidden dangers, the hidden danger identification results are analyzed to obtain the hidden danger categories and hidden danger location information of all hidden dangers, and based on the hidden danger categories and hidden danger location information, a hidden danger prompt message is generated and pushed. The hidden danger prompt message carries the hidden danger categories and hidden danger location information of all hidden dangers, such as "there is a hidden danger of foreign object entanglement in transmission line A."

[0116] When the hidden danger identification result indicates that there is no hidden danger, the transmission line is normal at this time, and a normal transmission line message is pushed, such as "all transmission lines are normal".

[0117] In the above embodiment, by pushing hidden danger prompt messages or normal transmission line messages to users based on hidden danger identification results, users can promptly know whether there are hidden dangers in the transmission lines in the transmission line environment image, so as to promptly repair the transmission lines with hidden dangers.

[0118] In an exemplary embodiment, to address the problem that current detection methods have low detection accuracy for long and slender transmission lines in complex scenarios, a cascaded model combination approach is adopted to first locate the transmission lines in the detection scenario, and then perform refined detection of various hidden dangers in the transmission lines, thereby achieving accurate identification of hidden dangers such as broken strands, scattered strands, and entanglement of foreign objects that may exist on the transmission lines.

[0119] Step S1: Acquire a transmission line environment image of an overhead transmission line through a camera installed at a fixed position or a camera mounted on a drone.

[0120] Step S2: The collected image is input into the RetinaNet model with an added slender object adaptive module to perform target detection of the transmission line.

[0121] RetinaNet is a single-stage object detection model that has high accuracy and robustness in detecting small objects in large backgrounds. The architecture of RetinaNet mainly consists of three parts: the base network, the feature pyramid network, and the classification and regression sub-networks.

[0122] The output of the base network will be used to generate feature maps of different scales, which will be used for subsequent target classification and location regression tasks.

[0123] The feature pyramid network is built on top of the basic network, using top-down paths and lateral connections to enhance the representation of high-level semantic information on feature maps of all scales. The feature pyramid network can effectively handle objects of different sizes and set prediction heads on each fused feature map, such as task heads for classification and regression, so that targets can be detected at each level, improving the accuracy of small object detection.

[0124] The classification subnetwork is responsible for predicting the category of the anchor point, and the regression subnetwork is responsible for predicting the offset of each detection box relative to its corresponding ground-truth bounding box.

[0125] The RetinaNet model can directly predict the location and category of the target from the input image. After adding the slender object adaptive module, the RetinaNet model can more accurately identify slender objects such as power lines. The output information of the RetinaNet model with the slender object adaptive module is the coordinates of the upper left corner and the lower right corner of the power line target box. According to the coordinate information of the power line target box, the image in the area is cropped out to form an image containing the power line target.

[0126] Step S3: Enlarge the image containing the power transmission line target to obtain a clear image containing the power transmission line target of the same size as the original image. The cropped image is usually thin and blurry, and needs to be restored by image restoration to obtain an image that is convenient for subsequent model processing.

[0127] Step S4: Input the image containing the transmission line target processed in step S3 into the Yolov5 model to identify hidden dangers of transmission line conductors. The architecture of Yolov5 includes several important parts, mainly the input layer, the skeleton network, the neck network and the head network. The input layer of the Yolov5 model supports image inputs of various sizes. The skeleton network is responsible for extracting high-level features from the input image, the neck network is responsible for enhancing and fusing features of different scales, and the head network is responsible for the final target classification and bounding box regression. The Yolov5 model identifies hidden dangers such as broken strands, scattered strands, and foreign body entanglement that may exist on the transmission line.

[0128] Step S5: Output the final result according to the recognition result. If there is a hidden danger, output "there is a hidden danger in the line" and output the corresponding hidden danger category, such as broken strands or foreign objects entangled, etc. If there is no hidden danger, output "the transmission line is normal".

[0129] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0130] Based on the same inventive concept, the embodiment of the present application also provides a transmission line hidden danger identification device for implementing the transmission line hidden danger identification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more transmission line hidden danger identification device embodiments provided below can refer to the limitations of the transmission line hidden danger identification method above, and will not be repeated here.

[0131] In an exemplary embodiment, Figure 4 As shown, a transmission line hidden danger identification device is provided, comprising: an image acquisition module 100, a target detection module 200, an image segmentation module 300, a size conversion module 400 and a hidden danger identification module 500, wherein:

[0132] An image acquisition module 100 is used to acquire an image of a power transmission line environment;

[0133] The target detection module 200 is used to perform target detection on the power transmission lines of different sizes in the power transmission line environment image, and obtain the power transmission line detection frames of different sizes in the power transmission line environment image and the power transmission line detection frame position information of each power transmission line detection frame;

[0134] An image segmentation module 300 is used to segment multiple transmission line detection frames and background information in the transmission line environment image based on the position information of all transmission line detection frames to obtain multiple transmission line images containing transmission lines;

[0135] The size conversion module 400 is used to update each power transmission line image to a target power transmission line image having the same size as the power transmission line environment image, and perform image restoration on the target power transmission line image to obtain an updated target power transmission line image;

[0136] The hidden danger identification module 500 is used to identify hidden dangers on multiple updated target transmission line images to obtain hidden danger identification results.

[0137] In one embodiment, the target detection module 200 is also used to perform target detection on transmission lines of different sizes in the transmission line environment image through a preloaded visual detection model, and obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame.

[0138] In one embodiment, the preloaded visual detection model includes a basic network, a feature pyramid network and a classification regression subnetwork; the target detection module 200 is also used to perform feature extraction of different scales on the transmission line environment image through the basic network to obtain multiple transmission line feature images; perform feature fusion processing on the multiple transmission line feature images through the feature pyramid network to obtain at least one first fused feature image; perform target detection on each first fused feature image through the classification regression subnetwork, generate multiple first object detection frames of different sizes on each first fused feature image, and detect the first category and the first object detection frame position information of each first object detection frame, use the first object detection frame with the first category of transmission line as the transmission line detection frame, and use the first object detection frame position information of the transmission line detection frame as the transmission line detection frame position information of the transmission line detection frame.

[0139] In one embodiment, the hidden danger identification module 500 is also used to identify hidden dangers in each updated target transmission line image through a preloaded image recognition model, and obtain the hidden danger category and hidden danger location information of each hidden danger in the updated target transmission line image; based on the hidden danger category and hidden danger location information of each hidden danger in all updated target transmission line images, generate a hidden danger identification result.

[0140] In one embodiment, the image recognition model includes an input layer, a skeleton network, a neck network and a head network; the hidden danger identification module 500 is also used to output the updated target transmission line image to the skeleton network through the input layer; extract at least one target image feature whose importance is higher than a preset importance threshold in the updated target transmission line image through the skeleton network; perform feature enhancement on at least one target image feature through the neck network, and perform feature fusion on the target image feature with the target image feature after feature enhancement to obtain a second fused feature image; perform target detection on the second fused feature image through the head network, generate a second object detection frame for each hidden danger in the second fused feature image, and detect the hidden danger category and hidden danger location information of each second object detection frame.

[0141] In one embodiment, the transmission line hidden danger identification device also includes a push module, which is used to obtain the hidden danger category and hidden danger location information of all hidden dangers from the hidden danger identification result when the hidden danger identification result indicates the existence of hidden dangers, and push hidden danger prompt messages based on the hidden danger category and hidden danger location information of all hidden dangers; when the hidden danger identification result indicates that there are no hidden dangers, push the transmission line normal message.

[0142] Each module in the above-mentioned power transmission line hidden danger identification device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0143] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as transmission line environment images. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying hidden dangers in a transmission line is implemented.

[0144] Those skilled in the art will understand that Figure 5 The structure shown in the figure is a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0145] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0146] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0147] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0148] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0149] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0150] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for identifying hidden dangers of a power transmission line, characterized in that: The method comprises: Acquire transmission line environment images; Performing target detection on the transmission lines of different sizes in the transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each of the transmission line detection frames; Based on the position information of all the power transmission line detection frames, segmenting the plurality of power transmission line detection frames and the background information in the power transmission line environment image to obtain a plurality of power transmission line images containing the power transmission lines; For each of the power transmission line images, updating the power transmission line image to a target power transmission line image having a size consistent with the power transmission line environment image, and performing image restoration on the target power transmission line image to obtain an updated target power transmission line image; Hidden danger identification is performed on the multiple updated target power transmission line images to obtain hidden danger identification results.

2. The method according to claim 1, characterized in that The performing target detection on the transmission lines of different sizes in the transmission line environment image to obtain the transmission line detection frames of different sizes in the transmission line environment image and the transmission line detection frame position information of each of the transmission line detection frames includes: Through the preloaded visual detection model, target detection is performed on the transmission lines of different sizes in the transmission line environment image to obtain transmission line detection frames of different sizes in the transmission line environment image and transmission line detection frame position information of each transmission line detection frame.

3. The method according to claim 2, characterized in that The preloaded visual detection model includes a basic network, a feature pyramid network, and a classification regression subnetwork; The method of performing target detection on the transmission lines of different sizes in the transmission line environment image by using the preloaded visual detection model to obtain the transmission line detection frames of different sizes in the transmission line environment image and the transmission line detection frame position information of each of the transmission line detection frames includes: Extracting features of different scales from the power transmission line environment image through the basic network to obtain a plurality of power transmission line feature images; Performing feature fusion processing on the plurality of power transmission line feature images through the feature pyramid network to obtain at least one first fused feature image; Perform target detection on each of the first fused feature images through the classification regression subnetwork, generate multiple first object detection frames of different sizes on each of the first fused feature images, detect the first category and the first object detection frame position information of each of the first object detection frames, use the first object detection frame whose first category is the power transmission line as the power transmission line detection frame, and use the first object detection frame position information of the power transmission line detection frame as the power transmission line detection frame position information of the power transmission line detection frame.

4. The method according to claim 1, characterized in that: The step of performing hidden danger identification on the plurality of updated target transmission line images to obtain hidden danger identification results includes: For each of the updated target transmission line images, identify hidden dangers on the updated target transmission line images by using a preloaded image recognition model to obtain the hidden danger category and hidden danger location information of each hidden danger in the updated target transmission line image; Based on the hidden danger category and hidden danger location information of each hidden danger in all updated target transmission line images, a hidden danger identification result is generated.

5. The method according to claim 4, characterized in that The image recognition model includes an input layer, a skeleton network, a neck network and a head network; The method of identifying hidden dangers on the updated target transmission line image by using the preloaded image recognition model to obtain the hidden danger category and hidden danger location information of each hidden danger in the updated target transmission line image includes: Outputting the updated target power transmission line image to the skeleton network through the input layer; extracting at least one target image feature having an importance higher than a preset importance threshold in the updated target power transmission line image through the skeleton network; Performing feature enhancement on at least one target image feature through the neck network, and performing feature fusion on the target image feature with the target image feature after the feature enhancement to obtain a second fused feature image; The head network performs target detection on the second fused feature image, generates a second object detection frame for each hidden danger in the second fused feature image, and detects the hidden danger category and hidden danger location information of each second object detection frame.

6. The method according to claim 2, characterized in that After performing hidden danger identification on the plurality of updated target transmission line images to obtain hidden danger identification results, the method further includes: In the case where the hidden danger identification result indicates that a hidden danger exists, obtaining hidden danger categories and hidden danger location information of all hidden dangers from the hidden danger identification result, and pushing hidden danger prompt messages according to the hidden danger categories and hidden danger location information of all hidden dangers; When the hidden danger identification result indicates that there is no hidden danger, a message that the power transmission line is normal is pushed.

7. A device for identifying hidden dangers of power transmission lines, characterized in that: The device comprises: An image acquisition module, used to acquire an image of the power transmission line environment; A target detection module, used to perform target detection on the power transmission lines of different sizes in the power transmission line environment image, and obtain power transmission line detection frames of different sizes in the power transmission line environment image and the power transmission line detection frame position information of each of the power transmission line detection frames; An image segmentation module, configured to segment a plurality of the transmission line detection frames and background information in the transmission line environment image based on the position information of all the transmission line detection frames, so as to obtain a plurality of transmission line images containing transmission lines; a size conversion module, for updating each of the power transmission line images into a target power transmission line image having the same size as the power transmission line environment image, and performing image restoration on the target power transmission line image to obtain an updated target power transmission line image; The hidden danger identification module is used to identify hidden dangers on the multiple updated target transmission line images to obtain hidden danger identification results.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images