Power transmission tower footing hidden danger identification method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a transmission tower footing hidden danger identification method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: performing feature extraction on a transmission tower footing image to obtain an initial to-be-detected feature image, performing layer-by-layer sampling on the initial to-be-detected feature image to obtain a to-be-detected feature image of each layer of network in a feature pyramid network, performing channel attention feature fusion and space attention feature fusion on the to-be-detected feature image of each level of network to obtain an attention feature image of each level of network; carrying out attention feature fusion on the attention feature image of each level of network and the attention feature images among different levels of networks, and updating the attention feature image of each level of network; and carrying out hidden danger identification on the attention feature images of all hierarchical networks. By adopting the method, the hidden danger of the transmission tower footing can be accurately identified.

Description

Technical Field

[0001] The present application relates to the field of power electronics technology, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for identifying hidden dangers in the foundation of a transmission tower. Background Art

[0002] As the social demand for electricity continues to rise, transmission towers, as an indispensable key infrastructure in the power transmission system, have a decisive influence on the safety and reliability of the entire transmission line due to their structural stability. In rainy and humid areas such as the south, frequent heavy rains and humid environments bring severe external environmental challenges to the base of transmission towers. For example, continuous ground waterlogging and uncontrolled growth of plant vines will cause the soil structure around the tower base to loosen, which may not only affect the stability of the tower base itself, but may also further threaten the operation safety of the entire transmission line. Therefore, it is particularly important to provide a method for identifying hidden dangers in the base of transmission towers.

[0003] In traditional technology, drones are usually used to identify hidden dangers in the base of transmission towers. Specifically, the intelligent detection system on the drone can carry high-definition cameras and advanced sensors to comprehensively shoot the area around the base of the transmission tower and capture high-quality drone patrol images. These images are then transmitted to the data processing center on the ground, where they are analyzed in detail by the intelligent detection system. Image recognition technology is used to determine whether there are safety hazards such as water accumulation and vine entanglement in the base of the transmission tower.

[0004] However, the current method of identifying hidden dangers in the foundation of transmission towers is not accurate enough. Summary of the invention

[0005] Based on this, it is necessary to provide an accurate method, device, computer equipment, computer-readable storage medium and computer program product for identifying hidden dangers in the foundation of a transmission tower in response to the above-mentioned technical problems.

[0006] In a first aspect, the present application provides a method for identifying hidden dangers of a transmission tower foundation, comprising:

[0007] Acquire the transmission tower base image;

[0008] Extracting features from the transmission tower base image to obtain an initial feature image to be detected, and inputting the initial feature image to be detected into a feature pyramid network;

[0009] The initial feature image to be detected is sampled layer by layer to obtain the feature image to be detected of each level of the network in the feature pyramid network, and the channel attention feature fusion and the spatial attention feature fusion are performed on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network;

[0010] Performing attention feature fusion on the attention feature image of each layer network and the attention feature images between different layers networks, and updating the attention feature image of each layer network;

[0011] Hidden danger identification is performed on the attention feature images of all hierarchical networks to obtain hidden danger identification results of the transmission tower base image.

[0012] In one embodiment, feature extraction is performed on the transmission tower base image to obtain an initial feature image to be detected, including:

[0013] Through the backbone network of the preloaded target detection model, feature extraction is performed on the base image of the transmission tower to obtain an initial feature image to be detected, wherein the backbone network includes a basic network and a residual network;

[0014] Through the backbone network of the preloaded target detection model, the transmission tower base image is feature extracted to obtain the initial feature image to be detected, including:

[0015] The basic network is used to extract features from the transmission tower base image to obtain an initial feature image to be detected;

[0016] The residual network is used to perform residual mapping on the initial feature image to be detected, and the initial feature image to be detected is updated.

[0017] In one embodiment, each level of the feature pyramid network includes a channel attention module and a spatial attention module; channel attention feature fusion and spatial attention feature fusion are performed on the feature image to be detected of each level of the network respectively to obtain an attention feature image of each level of the network, including:

[0018] For each layer of the network, the channel feature fusion is performed on the feature image to be detected through the channel attention module to obtain the channel attention feature image;

[0019] The spatial attention module is used to perform spatial feature fusion on the feature image to be detected to obtain a spatial attention feature image;

[0020] An attention feature image is generated according to the channel attention feature image and the spatial attention feature image.

[0021] In one embodiment, the channel attention module includes a fully connected layer, and channel features are fused on the feature image to be detected through channel attention to obtain a channel attention feature image, including:

[0022] Through the channel attention module, the feature image to be detected is processed by global average pooling to obtain the first channel feature image;

[0023] Through the channel attention module, the feature image to be detected is processed by global maximum pooling to obtain the second channel feature image;

[0024] The first channel feature image and the second channel feature image are fused through the fully connected layer to obtain a channel attention feature image.

[0025] In one embodiment, spatial feature fusion is performed on the feature image to be detected through spatial attention to obtain a spatial attention feature image, including:

[0026] Performing global average pooling processing on the feature image to be detected through spatial attention to obtain a first spatial feature image;

[0027] Performing global maximum pooling processing on the feature image to be detected through spatial attention to obtain a second spatial feature image;

[0028] Perform feature stitching on the first spatial feature image and the second spatial feature image according to channels to obtain a spatial stitching feature image;

[0029] The spatial splicing feature image is convolved to obtain the spatial attention feature image.

[0030] In one embodiment, the attention feature image of each layer of the network is fused with the attention feature images between different layers of the network to update the attention feature image of each layer of the network, including:

[0031] Perform feature sampling on the attention feature image of the Nth level network in the feature pyramid network to obtain a sampled attention feature image, where N is a positive integer;

[0032] The sampled attention feature image is feature fused with the attention feature image of the N-1th level network through the N-1th level network of the feature pyramid network, and the attention feature image of the N-1th level network is updated, wherein, in the feature pyramid network, the Nth level network is at the top of the N-1th level network.

[0033] In one embodiment, hidden dangers are identified on the attention feature images of all hierarchical networks to obtain hidden danger identification results of the transmission tower base image, including:

[0034] Through the pre-configured hidden danger anchor frame, hidden dangers are identified in the attention feature images of all levels of networks, and the hidden danger identification results of the transmission tower base image are obtained, wherein the hidden danger anchor frame is an anchor frame that matches the size of the actual hidden danger.

[0035] In a second aspect, the present application also provides a transmission tower base hidden danger identification device, comprising:

[0036] An image acquisition module, used to acquire an image of a transmission tower base;

[0037] A feature extraction module is used to extract features from the transmission tower base image to obtain an initial feature image to be detected, and input the initial feature image to be detected into a feature pyramid network;

[0038] The attention fusion module is used to sample the initial feature image to be detected layer by layer to obtain the feature image to be detected of each level of the feature pyramid network, and perform channel attention feature fusion and spatial attention feature fusion on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network;

[0039] A hierarchical fusion module is used to fuse the attention feature image of each layer of the network with the attention feature images between different layers of the network, and update the attention feature image of each layer of the network;

[0040] The hidden danger identification module is used to identify hidden dangers in the attention feature images of all levels of networks and obtain hidden danger identification results of the transmission tower base image.

[0041] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0042] Acquire the transmission tower base image;

[0043] Extract features from the transmission tower base image to obtain an initial feature image to be detected, and input the initial feature image to be detected into a feature pyramid network;

[0044] The initial feature image to be detected is sampled layer by layer to obtain the feature image to be detected of each level of the feature pyramid network, and the channel attention feature fusion and spatial attention feature fusion are performed on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network;

[0045] The attention feature image of each layer of the network is fused with the attention feature images between different layers of the network, and the attention feature image of each layer of the network is updated;

[0046] Hidden danger identification is performed on the attention feature images of all hierarchical networks to obtain hidden danger identification results of the transmission tower base image.

[0047] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0048] Acquire the transmission tower base image;

[0049] Extract features from the transmission tower base image to obtain an initial feature image to be detected, and input the initial feature image to be detected into a feature pyramid network;

[0050] The initial feature image to be detected is sampled layer by layer to obtain the feature image to be detected of each level of the feature pyramid network, and the channel attention feature fusion and spatial attention feature fusion are performed on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network;

[0051] The attention feature image of each layer of the network is fused with the attention feature images between different layers of the network, and the attention feature image of each layer of the network is updated;

[0052] Hidden danger identification is performed on the attention feature images of all hierarchical networks to obtain hidden danger identification results of the transmission tower base image.

[0053] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0054] Acquire the transmission tower base image;

[0055] Extract features from the transmission tower base image to obtain an initial feature image to be detected, and input the initial feature image to be detected into a feature pyramid network;

[0056] The initial feature image to be detected is sampled layer by layer to obtain the feature image to be detected of each level of the feature pyramid network, and the channel attention feature fusion and spatial attention feature fusion are performed on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network;

[0057] The attention feature image of each layer of the network is fused with the attention feature images between different layers of the network, and the attention feature image of each layer of the network is updated;

[0058] Hidden danger identification is performed on the attention feature images of all hierarchical networks to obtain hidden danger identification results of the transmission tower base image.

[0059] In the above-mentioned transmission tower foundation hidden danger identification method, device, computer equipment, computer-readable storage medium and computer program product, during the whole process, after the feature pyramid network samples the initial feature image to be detected layer by layer, channel attention feature fusion and spatial attention feature fusion are performed on the feature image to be detected at the same level, and then the attention feature image of each level of the network is fused with the attention feature images between different levels of networks, and the attention feature image of each level of the network is updated, so that more attention can be paid to important feature areas in the feature image to be detected, and the ability to capture details of the feature image to be detected is improved. Furthermore, since the features in the updated attention feature image are more significant, the accuracy of hidden danger identification for the attention feature images of all levels of networks is also higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 A diagram of an application environment of a method for identifying hidden dangers of a transmission tower foundation in one embodiment;

[0062] Figure 2 A schematic diagram of a flow chart of a method for identifying hidden dangers of a transmission tower foundation in one embodiment;

[0063] Figure 3 A schematic diagram of a flow chart of a method for identifying hidden dangers of a transmission tower foundation in another embodiment;

[0064] Figure 4 It is a flow chart of a method for identifying hidden dangers of a transmission tower foundation in a specific application embodiment;

[0065] Figure 5 It is a structural block diagram of a transmission tower base hidden danger identification device in one embodiment;

[0066] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are used to explain the present application and are not used to limit the present application.

[0068] The transmission tower foundation hidden danger identification method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 and the shooting device 106 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The shooting device 106 can be placed in a fixed position or carried on a drone.

[0069] The user triggers the transmission tower base hidden danger identification control on the terminal 102, and the terminal 102 responds to the trigger operation and controls the camera 106 placed at a fixed position to start up to take an image of the transmission tower base of the transmission tower, or controls the drone to take off according to a preset path, and controls the camera 106 carried by the drone to start up to take an image of the transmission tower base. The camera 106 transmits the captured transmission tower base image back to the terminal 102, and the terminal 102 uploads the transmission tower base image to the server 104, so as to identify the hidden dangers of the transmission tower base based on the transmission tower base image.

[0070] The server 104 obtains the transmission tower base image; performs feature extraction on the transmission tower base image to obtain an initial feature image to be detected, and inputs the initial feature image to be detected into the feature pyramid network; performs layer-by-layer sampling on the initial feature image to be detected to obtain the feature image to be detected of each level of the network in the feature pyramid network, and performs channel attention feature fusion and spatial attention feature fusion on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network; performs attention feature fusion on the attention feature image of each level of the network with the attention feature images between different levels of networks, and updates the attention feature image of each level of the network; performs hidden danger identification on the attention feature images of all levels of networks to obtain the hidden danger identification result of the transmission tower base image.

[0071] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0072] In an exemplary embodiment, Figure 2 As shown in the figure, a method for identifying hidden dangers of transmission tower foundation is provided. Figure 1 The server 104 in FIG. 1 is used as an example for explanation. Among them:

[0073] S100, acquiring a transmission tower base image.

[0074] Specifically, the user triggers the transmission tower base hidden danger identification control on the terminal, and the terminal responds to the trigger operation, controls the camera placed in a fixed position to start, so as to take the transmission tower base image of the transmission tower base, or controls the drone to take off according to a preset path, and controls the camera on the drone to start, so as to take the transmission tower base image of the transmission tower base. The camera transmits the captured transmission tower base image back to the terminal, and the terminal uploads the transmission tower base image to the server, so that the server identifies the hidden dangers of the transmission tower base based on the transmission tower base image. Generally, the server identifies the hidden dangers of the transmission tower base image by loading the target detection model stored in the database, such as the Yolov5 model, where YOLOv5 is an efficient target detection model.

[0075] In some embodiments, the image of the transmission pole tower base may also be captured in advance by a shooting device and stored locally in the terminal. When the user triggers the transmission pole tower base hidden danger identification control on the terminal, the terminal responds to the trigger operation and obtains the transmission pole tower base image required for identification locally to perform transmission pole tower base hidden danger identification.

[0076] S200, extracting features from the transmission tower base image to obtain initial feature images to be detected, and inputting the initial feature images to be detected into a feature pyramid network respectively.

[0077] Specifically, the architecture of the target detection model includes an input layer, a backbone network, a neck structure and a head structure. The input layer is used to obtain the image of the base of the transmission pole tower and send the image of the base of the transmission pole tower to the backbone network. The backbone network is used to extract features from the image of the base of the transmission pole tower to obtain the feature image to be detected.

[0078] In the process of the backbone network extracting features from the transmission tower base image, the features of the transmission tower base image are generally extracted through convolution operations. It can be considered that the backbone network of the target detection model is a deep convolutional network, whose main task is to extract high-level features from the transmission tower base image, and the backbone network can use cross-stage partial connections to reduce the amount of calculation and increase the training speed.

[0079] Furthermore, when the backbone network extracts feature images to be detected from the transmission tower base image, the backbone network inputs the extracted feature images to be detected into the feature pyramid network respectively.

[0080] In an exemplary embodiment, before extracting features from a transmission tower base image to obtain a plurality of feature images to be detected, the method further includes: performing image processing on the transmission tower base image, i.e. preprocessing the transmission tower base image into a uniform size, wherein the preprocessing includes scaling and padding to ensure the consistency of input data and adapt to network design.

[0081] S300, sampling the initial feature image to be detected layer by layer to obtain the feature image to be detected of each level of the network in the feature pyramid network, and performing channel attention feature fusion and spatial attention feature fusion on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network.

[0082] Specifically, the neck network in the target detection model includes a feature pyramid network. The feature pyramid network integrates high-level contour features and low-level detailed features upwards. Through the feature pyramid network, the initial feature image to be detected is sampled layer by layer to obtain the feature image to be detected at each level of the feature pyramid network. For example, the steps of layer-by-layer sampling are as follows:

[0083] The lowest layer, such as the initial feature image to be detected in the first layer, is downsampled, and the initial feature image to be detected in the lowest layer is used as the feature image to be detected in the lowest layer, and the downsampled initial feature image to be detected is used as the feature image to be detected in the upper layer, that is, the second layer, and the feature image to be detected in the second layer is downsampled again, and the downsampled initial feature image to be detected is used as the feature image to be detected in the upper layer, that is, the third layer, and so on. The cycle is repeated until the topmost feature image to be detected is obtained. In addition, the topmost feature image to be detected can also be downsampled to obtain additional feature images to be detected. It can be seen that the layer-by-layer sampling process is also the forward propagation process.

[0084] Next, for each level of the feature pyramid network, CBAM (Convolutional Block Attention Module) is added to each level of the network to perform channel attention feature fusion and spatial attention feature fusion on the feature image to be detected. Among them, CBAM is an attention mechanism for enhancing convolutional neural networks. By sequentially integrating channel attention and spatial attention, it systematically focuses on the important information in the input feature image.

[0085] Through the CBAM module in each level of the network, the channel attention feature fusion and the spatial attention feature fusion are performed on the feature image to be detected to obtain the attention feature image of each level of the network.

[0086] S400, performing attention feature fusion on the attention feature image of each layer of the network and the attention feature images between different layers of the network, and updating the attention feature image of each layer of the network.

[0087] Specifically, the feature pyramid network actually involves the path aggregation network. The feature pyramid network integrates high-level contour features with low-level detailed features. The path aggregation network makes improvements on this basis and enhances the downward path of features (from the top to the bottom) to improve the semantic expression of low-level features. Therefore, the feature pyramid network can be used to achieve feature fusion of attention feature images between different levels, that is, to fuse the attention feature images of each level of the network with the attention feature images between different levels of the network, and update the attention feature images of each level of the network.

[0088] S500, performing hidden danger identification on the attention feature images of all hierarchical networks to obtain hidden danger identification results of the transmission tower base image.

[0089] Specifically, the head structure of the target detection model is a structure for object prediction, including the location of the predicted bounding box, the confidence of the object, and the category probability. A corresponding head structure is set on each layer of the network, and the attention feature image of each layer of the network is input into the corresponding head structure to generate the predicted bounding box, predicted bounding box location, and predicted bounding box category of the hidden danger in the transmission tower base image.

[0090] In the above-mentioned transmission tower base hidden danger identification method, during the whole process, after the feature pyramid network samples the initial feature image to be detected layer by layer, the channel attention feature fusion and the spatial attention feature fusion are performed on the feature image to be detected at the same level, and then the attention feature image of each level of the network is fused with the attention feature images between different levels of networks, and the attention feature image of each level of the network is updated, so that more attention can be paid to the important feature areas in the feature image to be detected, and the ability to capture details of the feature image to be detected is improved. Furthermore, since the features in the updated attention feature image are more significant, the accuracy of hidden danger identification for the attention feature images of all levels of networks is also higher.

[0091] In an exemplary embodiment, feature extraction is performed on a transmission tower base image to obtain an initial feature image to be detected, including:

[0092] Through the backbone network of the preloaded target detection model, feature extraction is performed on the base image of the transmission tower to obtain an initial feature image to be detected, wherein the backbone network includes a basic network and a residual network;

[0093] Through the backbone network of the preloaded target detection model, the transmission tower base image is feature extracted to obtain the initial feature image to be detected, including:

[0094] The basic network is used to extract features from the transmission tower base image to obtain an initial feature image to be detected; the residual network is used to perform residual mapping processing on the initial feature image to be detected to update the initial feature image to be detected.

[0095] Specifically, the preloaded target detection model may be a Yolov5 model, and features of the transmission tower base image are extracted through the backbone network of the preloaded Yolov5 model to obtain a feature image to be detected.

[0096] This application introduces more residual network connections on the original basic network of the backbone network of the Yolov5 model. The residual connection allows the network layer to learn the residual between the input and output, so that information can be directly transferred from the previous layer to the next layer without being affected by the subsequent layers. It is used to solve the degradation problem in deep neural networks. The degradation problem means that as the number of network layers increases, the performance of the network does not improve but decreases. In other words, the backbone network at this time includes the basic network and the residual network. Through the basic network and the residual network in the backbone network, the image of the base of the transmission tower is feature extracted to obtain the feature image to be detected.

[0097] Among them, the basic network includes multiple convolutional layers, batch normalization layers and activation layers, which are used to accurately extract features of the transmission tower base image to obtain the initial feature image to be detected. Then, the initial feature image to be detected is processed by residual mapping through multiple residual networks, so that the feature image to be detected can learn the feature information of the input transmission tower base image to alleviate the degradation problem in the deep neural network and obtain the updated initial feature image to be detected.

[0098] In this embodiment, by using a deeper residual network structure, the model's ability to extract key features of accumulated water and vines and the stability of training are enhanced, so that important feature information in the transmission tower base image can be extracted.

[0099] In an exemplary embodiment, Figure 3 As shown, each level of the feature pyramid network includes a channel attention module and a spatial attention module; S300 includes:

[0100] S320, sampling the initial feature image to be detected layer by layer through the feature pyramid network to obtain the feature image to be detected at each level of the feature pyramid network.

[0101] S340, for each level of the network, channel features are fused on the feature image to be detected through channel attention to obtain a channel attention feature image.

[0102] S360, performing spatial feature fusion on the feature image to be detected through spatial attention to obtain a spatial attention feature image.

[0103] S380, generating an attention feature image according to the channel attention feature image and the spatial attention feature image.

[0104] Specifically, the feature image to be detected extracted by the backbone network is input into the neck network for feature fusion. The neck network includes a feature pyramid network, which is a multi-level network. The feature pyramid network samples the initial feature image to be detected layer by layer through a bottom-up approach, so that the feature image to be detected corresponding to the initial feature image to be detected is generated in each level of the feature pyramid network.

[0105] For each level of the feature pyramid network, the channel attention feature fusion and spatial attention feature fusion are performed on the feature image to be detected through the channel attention module and the spatial attention module to obtain the attention feature image.

[0106] Among them, the purpose of the channel attention module is to generate a channel attention feature image to highlight those channels with more important information, and the spatial attention module generates a spatial attention feature image to further refine the importance of each position of the feature map.

[0107] That is to say, the channel attention feature image is fused by channel attention to obtain the channel attention feature image, and the spatial attention feature image is fused by spatial attention to obtain the spatial attention feature image. Finally, the attention feature image is generated by fusing the channel attention feature image with the spatial attention feature image, that is, the final attention feature image is obtained by element-by-element multiplication of the channel attention feature image and the spatial attention feature image.

[0108] In this embodiment, the introduction of two attention structures, namely the channel attention module and the spatial attention module, allows the model to focus on more important feature areas, such as the texture of vines and the reflection of accumulated water, thereby improving the model's ability to capture details and making subsequent hidden danger identification more accurate.

[0109] In an exemplary embodiment, the channel attention module includes a fully connected layer, and channel features are fused on the feature image to be detected through channel attention to obtain a channel attention feature image, including:

[0110] The feature image to be detected is processed by global average pooling through channel attention to obtain the first channel feature image; the feature image to be detected is processed by global maximum pooling through channel attention to obtain the second channel feature image; the first channel feature image and the second channel feature image are feature fused through the fully connected layer to obtain the channel attention feature image.

[0111] Specifically, through channel attention, global average pooling and global maximum pooling are performed on the feature image to be detected to aggregate spatial information, and the statistical information of the feature map is summarized from different angles to obtain the first channel feature image after global average pooling and the second channel feature image after global maximum pooling. Among them, the global maximum pooling will select the maximum value in a specific local window (for example, 2x2) as the output value of the window, and the average pooling will take the average value in the specified window, and use this average value to represent the features of the window, so as to compress the spatial size of the feature image to be detected, which is convenient for the fully connected layer to learn channel features.

[0112] Furthermore, the two feature images obtained after pooling are processed by a shared fully connected layer, and the features of the channel dimension and the importance of each channel are learned through the fully connected layer. Finally, the two feature images output by the channel are feature fused, that is, added, to generate the weight of each channel, and weighted based on the weight of each channel to generate a channel attention feature image.

[0113] In this embodiment, by setting a fully connected layer in the channel attention module, the importance of each channel in the first channel feature image after global average pooling processing and the second channel feature image after global maximum pooling processing can be learned based on the fully connected layer to highlight those channels with more important information and improve the accuracy of hidden danger identification.

[0114] In an exemplary embodiment, spatial feature fusion is performed on the feature image to be detected through spatial attention to obtain a spatial attention feature image, including:

[0115] Performing global average pooling processing on the feature image to be detected through spatial attention to obtain a first spatial feature image; performing global maximum pooling processing on the feature image to be detected through spatial attention to obtain a second spatial feature image; performing feature splicing on the first spatial feature image and the second spatial feature image according to channels to obtain a spatial splicing feature image; performing convolution processing on the spatial splicing feature image to obtain a spatial attention feature image.

[0116] Specifically, the spatial attention module aggregates the channel dimension of the feature map by using the average value of the channel information and the maximum value of the channel information in the channel dimension, so as to aggregate and compress the channel dimension, so as to facilitate learning the spatial features of the image. That is, the feature image to be detected is processed by global average pooling through spatial attention to obtain the first spatial feature image, and the feature image to be detected is processed by global maximum pooling through spatial attention to obtain the second spatial feature image. The first spatial feature image and the second spatial feature image are two two-dimensional images. The two two-dimensional images are stacked, that is, the first spatial feature image and the second spatial feature image are spliced ​​according to the channel features to obtain a spatial splicing feature image. Finally, the spatial splicing feature image is convolved through a convolution layer to output a two-dimensional spatial attention feature image, which obtains the weight of each spatial position through an activation function.

[0117] In this embodiment, the feature image to be detected is subjected to global average pooling and global maximum pooling through spatial attention, and the first spatial feature image and the second spatial feature image obtained are feature spliced ​​according to channels, and then convolution processing is performed, so that the spatial attention feature image can be accurately obtained, so that in the subsequent hidden danger identification process, more attention can be paid to spatial areas with more important information, and the accuracy of hidden danger identification is higher.

[0118] In an exemplary embodiment, the attention feature images of all hierarchical networks are subjected to feature fusion between hierarchical networks, and the attention feature images of each hierarchical network are updated, including:

[0119] Feature sampling is performed on the attention feature image of the Nth level network in the feature pyramid network to obtain a sampled attention feature image, where N is a positive integer; through the N-1th level network of the feature pyramid network, the sampled attention feature image is feature fused with the attention feature image of the N-1th level network to update the attention feature image of the N-1th level network.

[0120] Among them, in the feature pyramid network, the Nth level network is at the top of the N-1th level network, that is, the 1st level network is the bottom network of the feature pyramid network. The larger the number of the level network, the closer it is to the top layer.

[0121] Specifically, the feature pyramid network can be improved by fusing high-level contour features with low-level detailed features, that is, adding a path aggregation network and fusing the path aggregation network with the feature pyramid network, thereby enhancing the downward path of features (from top to bottom) to improve the semantic expressiveness of low-level features.

[0122] Specifically, when the feature image to be detected is input into the feature pyramid network, a feature image to be detected can be generated in each level of the feature pyramid network through a bottom-up sampling method. Then, a dual fusion of channel attention and spatial attention is performed on each feature image to be detected to obtain an attention feature image.

[0123] Next, by means of top-down and horizontal connections, the attention feature image of each level of the network is fused with the attention feature images between different levels of the network, and the attention feature image of each level of the network is updated. In other words, the attention feature image of the Nth level network in the feature pyramid network is feature sampled to obtain the sampled attention feature image, where N is a positive integer; through horizontal connections, the sampled attention feature image is feature fused with the attention feature image of the N-1th level network, and the attention feature image of the N-1th level network is updated. The purpose of the horizontal connection is to change the number of channels so that the number of channels of the feature images of the previous and next two layers is the same.

[0124] In this embodiment, the attention feature images between different levels of networks are accurately fused through top-down and lateral connection methods to improve the accuracy of hidden danger identification.

[0125] In an exemplary embodiment, hidden dangers are identified on the attention feature images of all hierarchical networks to obtain hidden danger identification results of the transmission tower base image, including:

[0126] Through the pre-configured hidden danger anchor frame, hidden dangers are identified in the attention feature images of all levels of networks, and the hidden danger identification results of the transmission tower base image are obtained.

[0127] Among them, the hidden danger anchor frame is an anchor frame that matches the size of the actual hidden danger.

[0128] Specifically, the head structure usually sets multiple anchor frames for each predicted position, and each anchor frame is responsible for predicting an object of a specific size. This method adjusts the size and proportion of the anchor frame according to the typical appearance of hidden dangers such as water accumulation and vines. The hidden danger anchor frame obtained after adjustment can ensure that these targets that need to be detected can be matched more accurately. Through the pre-configured hidden danger anchor frame, the attention feature images of all hierarchical networks are used to identify hidden dangers, and the hidden danger identification results of the transmission tower base image are obtained. If there are hidden dangers, the hidden dangers such as vines on the transmission tower base and the surrounding water accumulation will be selected according to the pre-set anchor frame.

[0129] In this embodiment, by adjusting the size and proportion of the anchor frame to match the size of the actual hidden danger, the adjusted hidden danger anchor frame can ensure that it can more accurately match the targets that need to be detected, so that the hidden dangers of the transmission tower base can be identified more accurately.

[0130] In an exemplary embodiment, Figure 4 As shown in the figure, the method for identifying hidden dangers of transmission tower foundation includes:

[0131] Step S1: The drone camera obtains the image of the base of the transmission tower.

[0132] Step S2: The collected transmission tower base image is input into the Yolov5 model. YOLOv5 is an efficient target detection model, and its overall architecture can be divided into the input layer, backbone network, neck structure and head structure. The input image first enters the input layer, where the image will be preprocessed to a uniform size. The preprocessing includes scaling and padding to ensure the consistency of the input data and adapt to the network design.

[0133] Step S3: After the transmission tower base image is processed in the input layer, a convolution operation is performed in the backbone network to extract features to obtain the initial feature image to be detected. The backbone network of YOLOv5 is a deep convolutional network that uses cross-stage partial connections to reduce the amount of calculation and increase the training speed. It consists of multiple convolutional layers, batch normalization layers, and activation layers. The main task of the backbone network is to extract high-level features from the image. This method introduces more residual network connections to the original structure of the Yolov5 backbone network to solve the degradation problem in deep neural networks. By using a deeper residual network structure, the model's ability to extract key features of waterlogging and vines and the stability of training are enhanced.

[0134] Step S4: The initial feature image to be detected extracted by the backbone network is input into the neck structure, that is, the feature pyramid network for feature fusion. The feature pyramid network also requires the participation of the path aggregation network. The feature pyramid network upwardly fuses high-level contour features with low-level detailed features. The path aggregation network makes improvements on this basis and enhances the downward path of the feature (from the top to the bottom) to improve the semantic expression of the low-level features.

[0135] This method adds an attention module (CBAM) to each layer of the feature pyramid, which is an attention mechanism used to enhance convolutional neural networks. It systematically focuses on important information in the input feature map by sequentially integrating channel attention and spatial attention.

[0136] The purpose of the channel attention module is to generate a channel attention feature image to highlight those channels with more important information. The channel attention module aggregates spatial information through two strategies: global average pooling and global maximum pooling. It summarizes the statistical information of the feature map from different angles. The pooled features are processed through a shared fully connected layer and added together. Finally, the weight of each channel is generated to generate the final channel attention feature image based on the weight.

[0137] The spatial attention module further refines the importance of each position in the feature map. The spatial attention module aggregates the channel dimension of the feature map by using the pooled average and pooled maximum values ​​of the channel information in the channel dimension to form two 2D images. The two 2D images are stacked and processed through a convolution layer to output a spatial attention feature image, which uses an activation function to obtain the weight of each spatial position.

[0138] Based on the channel attention feature image and the spatial attention feature image, an attention feature image is generated. The introduction of the attention mechanism allows the model to focus on more important feature areas, such as the texture of vines and the reflection of water, improving the model's ability to capture details.

[0139] Furthermore, it is also necessary to cleverly integrate the high-level contour features and low-level detailed features in different levels of the feature pyramid network through top-down logic and lateral connections in the feature pyramid network, and update the attention feature image of each level of the network.

[0140] Step S5: The attention feature image fused with the neck structure features is input into the head structure for final prediction. The head structure of Yolov5 is responsible for the final object detection task, including the location of the predicted bounding box, the confidence of the object, and the category probability. The head usually sets multiple anchor frames for each predicted position, and each anchor frame is responsible for predicting an object of a specific size. This method adjusts the size and proportion of the anchor frame according to the typical appearance of accumulated water and vines to ensure that these targets can be matched more accurately. The head structure gives the final prediction result. If there are hidden dangers such as vines and surrounding accumulated water, the vines on the base of the transmission tower and the surrounding accumulated water will be selected.

[0141] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0142] Based on the same inventive concept, the embodiment of the present application also provides a transmission tower base hidden danger identification device for implementing the transmission tower base hidden danger identification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the transmission tower base hidden danger identification device provided below can refer to the limitations of the transmission tower base hidden danger identification method above, and will not be repeated here.

[0143] In an exemplary embodiment, Figure 5 As shown, a transmission tower base hidden danger identification device is provided, comprising: an image acquisition module 100, a feature extraction module 200, an attention fusion module 300, a hierarchical fusion module 400 and a hidden danger identification module 500, wherein:

[0144] An image acquisition module 100 is used to acquire an image of a transmission tower base;

[0145] The feature extraction module 200 is used to extract features from the transmission tower base image to obtain an initial feature image to be detected, and input the initial feature image to be detected into a feature pyramid network;

[0146] The attention fusion module 300 is used to sample the initial feature image to be detected layer by layer to obtain the feature image to be detected of each level of the network in the feature pyramid network, and perform channel attention feature fusion and spatial attention feature fusion on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network;

[0147] A hierarchical fusion module 400 is used to fuse the attention feature image of each hierarchical network with the attention feature images between different hierarchical networks, and update the attention feature image of each hierarchical network;

[0148] The hidden danger identification module 500 is used to identify hidden dangers in the attention feature images of all hierarchical networks and obtain hidden danger identification results of the transmission tower base image.

[0149] In one embodiment, the feature extraction module 200 is also used to perform feature extraction on the transmission tower base image through the backbone network of the preloaded target detection model to obtain an initial feature image to be detected, wherein the backbone network includes a basic network and a residual network; the feature extraction module 200 is also used to perform feature extraction on the transmission tower base image through the basic network to obtain an initial feature image to be detected; and perform residual mapping processing on the initial feature image to be detected through the residual network to update the initial feature image to be detected.

[0150] In one embodiment, each level of the feature pyramid network includes a channel attention module and a spatial attention module; the attention fusion module 300 is also used to perform channel feature fusion on the feature image to be detected through the channel attention module for each level of the network to obtain a channel attention feature image; perform spatial feature fusion on the feature image to be detected through the spatial attention module to obtain a spatial attention feature image; generate an attention feature image based on the channel attention feature image and the spatial attention feature image.

[0151] In one embodiment, the channel attention module includes a fully connected layer, and the attention fusion module 300 is also used to perform global average pooling processing on the feature image to be detected through the channel attention module to obtain a first channel feature image; perform global maximum pooling processing on the feature image to be detected through the channel attention module to obtain a second channel feature image; perform feature fusion on the first channel feature image and the second channel feature image through the fully connected layer to obtain a channel attention feature image.

[0152] In one embodiment, the attention fusion module 300 is also used to perform global average pooling processing on the feature image to be detected through spatial attention to obtain a first spatial feature image; perform global maximum pooling processing on the feature image to be detected through spatial attention to obtain a second spatial feature image; perform feature splicing on the first spatial feature image and the second spatial feature image according to channels to obtain a spatial splicing feature image; perform convolution processing on the spatial splicing feature image to obtain a spatial attention feature image.

[0153] In one embodiment, the hierarchical fusion module 400 is also used to perform feature sampling on the attention feature image of the Nth hierarchical network in the feature pyramid network to obtain a sampled attention feature image, wherein N is a positive integer; through the N-1th hierarchical network of the feature pyramid network, the sampled attention feature image is feature fused with the attention feature image of the N-1th hierarchical network to update the attention feature image of the N-1th hierarchical network, wherein in the feature pyramid network, the Nth hierarchical network is at the top of the N-1th hierarchical network.

[0154] In one embodiment, the hidden danger identification module 500 is also used to identify hidden dangers in the attention feature images of all hierarchical networks through a pre-configured hidden danger anchor frame to obtain hidden danger identification results of the transmission tower base image, wherein the hidden danger anchor frame is an anchor frame that matches the size of the actual hidden danger.

[0155] Each module in the transmission tower foundation hidden danger identification device can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0156] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as transmission tower base images. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying hidden dangers in the base of a transmission tower is implemented.

[0157] Those skilled in the art will understand that Figure 6 The structure shown in the figure is a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0159] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0160] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0162] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0163] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for identifying hidden dangers of a transmission tower foundation, characterized in that: The method comprises: Acquire the transmission tower base image; Extracting features from the transmission tower base image to obtain an initial feature image to be detected, and inputting the initial feature image to be detected into a feature pyramid network; The initial feature image to be detected is sampled layer by layer to obtain the feature image to be detected of each level of the network in the feature pyramid network, and the channel attention feature fusion and the spatial attention feature fusion are performed on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network; Performing attention feature fusion on the attention feature image of each layer network and the attention feature images between different layers networks, and updating the attention feature image of each layer network; Hidden danger identification is performed on the attention feature images of all hierarchical networks to obtain hidden danger identification results of the transmission tower base image.

2. The method according to claim 1, characterized in that: The step of extracting features from the transmission tower base image to obtain an initial feature image to be detected includes: The transmission tower base image is subjected to feature extraction through a preloaded target detection model backbone network to obtain an initial feature image to be detected, wherein the backbone network includes a basic network and a residual network; The backbone network of the preloaded target detection model is used to extract features from the transmission tower base image to obtain an initial feature image to be detected, including: Extracting features from the base image of the transmission tower through the basic network to obtain an initial feature image to be detected; The residual network is used to perform residual mapping processing on the initial feature image to be detected, so as to update the initial feature image to be detected.

3. The method according to claim 1, characterized in that Each level of the feature pyramid network includes a channel attention module and a spatial attention module; the channel attention feature fusion and the spatial attention feature fusion are respectively performed on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network, including: For each level of the network, channel feature fusion is performed on the feature image to be detected through the channel attention module to obtain a channel attention feature image; Performing spatial feature fusion on the feature image to be detected by the spatial attention module to obtain a spatial attention feature image; An attention feature image is generated according to the channel attention feature image and the spatial attention feature image.

4. The method according to claim 3, characterized in that The channel attention module includes a fully connected layer, and the channel feature fusion of the feature image to be detected is performed through the channel attention to obtain a channel attention feature image, including: Performing global average pooling processing on the feature image to be detected by the channel attention module to obtain a first channel feature image; Performing global maximum pooling processing on the feature image to be detected by the channel attention module to obtain a second channel feature image; The first channel feature image and the second channel feature image are feature fused through a fully connected layer to obtain a channel attention feature image.

5. The method according to claim 3, characterized in that: The step of performing spatial feature fusion on the feature image to be detected by using the spatial attention to obtain a spatial attention feature image includes: Performing global average pooling processing on the feature image to be detected by using the spatial attention to obtain a first spatial feature image; Performing global maximum pooling processing on the feature image to be detected through the spatial attention to obtain a second spatial feature image; Perform feature stitching on the first spatial feature image and the second spatial feature image according to channels to obtain a spatial stitching feature image; The spatial stitching feature image is convolved to obtain a spatial attention feature image.

6. The method according to claim 1, characterized in that The step of fusing the attention feature image of each level of the network with the attention feature images between different levels of the network, and updating the attention feature image of each level of the network, includes: Performing feature sampling on the attention feature image of the Nth level network in the feature pyramid network to obtain a sampled attention feature image, where N is a positive integer; The sampled attention feature image is feature fused with the attention feature image of the N-1-th level network through the N-1-th level network of the feature pyramid network, and the attention feature image of the N-1-th level network is updated, wherein, in the feature pyramid network, the N-th level network is at the top of the N-1-th level network.

7. The method according to claim 1, characterized in that The hidden danger identification is performed on the attention feature images of all hierarchical networks to obtain the hidden danger identification result of the transmission tower base image, including: Through the pre-configured hidden danger anchor frame, hidden danger identification is performed on the attention feature images of all hierarchical networks to obtain the hidden danger identification result of the transmission tower base image, wherein the hidden danger anchor frame is an anchor frame that matches the size of the actual hidden danger.

8. A transmission tower foundation hidden danger identification device, characterized in that: The device comprises: An image acquisition module, used to acquire an image of a transmission tower base; A feature extraction module is used to extract features from the transmission tower base image to obtain an initial feature image to be detected, and input the initial feature image to be detected into a feature pyramid network; An attention fusion module is used to sample the initial feature image to be detected layer by layer to obtain the feature image to be detected of each level of the network in the feature pyramid network, and perform channel attention feature fusion and spatial attention feature fusion on the feature image to be detected of each level of the network to obtain the attention feature image of each level of the network; A hierarchical fusion module, used to fuse the attention feature image of each hierarchical network with the attention feature images between different hierarchical networks, and update the attention feature image of each hierarchical network; The hidden danger identification module is used to identify hidden dangers in the attention feature images of all levels of networks and obtain hidden danger identification results of the transmission tower base image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images