Multi-target detection method, device and equipment for infrared image of power transmission line and medium
By introducing global context attention and context enhancement modules into the YOLOv5s algorithm, combined with the Wise-IoU v3 loss function, the problems of low detection accuracy and degradation in infrared image detection of transmission lines are solved, and high for complex backgrounds and small goals are achieved Accuracy identification.
Patent Information
- Application Number
- CN202411162702.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-08-22
AI Technical Summary
Traditional infrared image detection methods for power transmission lines have problems of low detection accuracy and degradation in complex backgrounds and practical applications, especially inaccurate identification of targets of different sizes and numbers, resulting in false detection and missed detection.
Using the improved YOLOv5s algorithm, by introducing a global context attention mechanism in the Backbone part and adding multiple context enhancement modules to the Neck part, combining Wise-IoU v3 loss function, the feature extraction and aggregation capabilities of the model are improved, and the detection of small targets is improved. Accuracy and robustness.
It significantly improves the accuracy and robustness of infrared image detection of transmission lines, and can more accurately identify insulators, overhanging wire clips, tension-resistant wire clips, shock-proof hammers, pressure equalization rings and fault heating points, improving the model's complex background Detection performance.
Smart Images

Figure CN119068308A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of infrared image recognition technology, and in particular to a multi-target detection method, device, equipment and medium for infrared images of power transmission lines. Background Art
[0002] The traditional inspection of power transmission lines in power systems mainly relies on manual inspections, which is not only time-consuming and costly, but also has safety issues. In recent years, the use of drones in conjunction with infrared thermal imagers to inspect transmission lines has gradually become a new inspection method. Infrared thermal imagers can accurately capture the heat distribution of equipment, help to discover areas of abnormal heat in equipment in advance, and identify possible problems. Drones equipped with infrared thermal imagers can accurately obtain thermal images of transmission lines in real time, thereby evaluating the operating status of transmission lines. In addition, with the continuous development of artificial intelligence, drones are equipped with deep learning algorithms to monitor power equipment on transmission lines. In particular, the outstanding performance of the YOLO (You Only Look Once, YOLO) algorithm in the field of target detection makes it a powerful tool for processing images.
[0003] In the field of target detection, target detection algorithms based on deep learning are mainly divided into two categories. One is the single-stage target detection algorithm represented by the YOLO series of algorithms and the SSD (Single Shot MultiBox Detector, SSD) algorithm, and the other is the two-stage target detection algorithm represented by the R-CNN (Region based Convolutional Neural Network, region-based convolutional neural network) series of algorithms. The advantages of the single-stage detection algorithm of the YOLO series are high real-time detection and fast detection speed. When processing the input image, it will directly identify and classify each area of the image. The two-stage target detection algorithm mainly refers to the model dividing the target detection task into two stages. The first stage is to generate candidate regions, and the second stage is to classify and regress the generated candidate regions. The advantage is that the detection accuracy is high, but the speed is slow. Therefore, the YOLO algorithm is currently mainly used in the industrial detection field for target detection.
[0004] In order to improve the detection accuracy of the YOLO series of algorithms, the first existing technology introduces a bidirectional feature pyramid network (BiFPN) to replace the original feature pyramid network in YOLOv5 to promote the free exchange of information and the fusion of features, thereby improving the accuracy of the model. The second existing technology builds an integrated model based on the YOLOv5s and YOLOv5x algorithms, and introduces the weighted boxes fusion (WBF) reasoning technology, so that the model has a good recognition effect on the tower under complex backgrounds, different lighting and weather conditions, and shows high robustness. The third existing technology proposes an improved YOLOv5 algorithm for multi-target detection. Its main improvements include using ShuffleNetv2 as the main feature extraction structure to reduce the amount of network parameters, and replacing the Bottleneck CSP module in the PANet network with the LightCSP module to accelerate feature fusion. Based on YOLOv5, the existing technology 4 adopts the lightweight Ghost convolution technology to replace the traditional convolution operation, and replaces the original feature extraction network with the repeatedly weighted BiFPN network to achieve rapid detection of insulator defects.
[0005] However, the above existing transmission line infrared image detection technologies all face some challenges: First, severe background interference or complex backgrounds lead to reduced image quality, which makes it difficult for traditional detection methods to fully utilize image information, affects the model's accurate expression of features, and is prone to false detection and missed detection. Secondly, when faced with targets of different sizes and numbers, traditional models tend to identify large-sized and large-number easy-to-classify samples, while ignoring small-sized, far-distance, and small-number difficult-to-classify samples. As a result, the traditional model has significant effects when performing algorithm simulation on the PC side, but the performance of the model in actual applications will be greatly reduced, which needs to be solved urgently. Summary of the invention
[0006] The present invention provides a multi-target detection method, device, equipment and medium for infrared images of power transmission lines to solve the problems of low detection accuracy and performance degradation in practical applications of traditional detection methods, and improve the accuracy and robustness of target detection.
[0007] The first aspect of the present invention provides a multi-target detection method for infrared images of power transmission lines, the method using an improved YOLOv5s algorithm, the improved YOLOv5s algorithm introduces a global context attention mechanism in the Backbone part of the original YOLOv5s algorithm, and adds multiple context enhancement modules in the Neck part of the original YOLOv5s algorithm, wherein the method comprises the following steps:
[0008] Acquire infrared images of the power transmission lines to be inspected;
[0009] Based on the Backbone part of the improved YOLOv5s algorithm, the attention weight feature information of the infrared image of the power transmission line to be detected is calculated, and the global feature relationship information is obtained according to the initial high-dimensional information of the infrared image of the power transmission line to be detected and the attention weight feature information, and the global feature relationship information is processed, and the global important information of the enhanced image is obtained according to the processed global feature relationship information and the initial high-dimensional information of the infrared image of the power transmission line to be detected;
[0010] Based on the Neck part of the improved YOLOv5s algorithm, feature aggregation is performed on the global important information of the enhanced image to obtain aggregated image information, and based on the Head part of the improved YOLOv5s algorithm, a target detection result is obtained according to the aggregated image information.
[0011] According to one embodiment of the present invention, the target detection result includes at least one of the insulator position and insulator confidence, the suspension wire clamp position and suspension wire clamp confidence, the tension wire clamp position and tension wire clamp confidence, the shock-absorbing hammer position and shock-absorbing hammer confidence, the equalizing ring position and equalizing ring confidence, the fault hot spot position and the fault hot spot confidence.
[0012] According to an embodiment of the present invention, the Head part based on the improved YOLOv5s algorithm obtains the target detection result according to the aggregated image information, including:
[0013] A detection frame is generated by using the Head part of the improved YOLOv5s algorithm, and classification, positioning and confidence scoring are performed based on the detection frame and the aggregated image information to obtain the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, anti-vibration hammer position, anti-vibration hammer confidence, grading ring position, grading ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected;
[0014] The target detection result is obtained according to the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, shock-absorbing hammer position, shock-absorbing hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected.
[0015] According to an embodiment of the present invention, the Neck part based on the improved YOLOv5s algorithm performs feature aggregation on the important global information of the enhanced image to obtain aggregated image information, including:
[0016] Based on the multiple context enhancement modules, adaptively calculate the global important information of the enhanced image to obtain multiple adaptive weights;
[0017] The aggregated image information is obtained by calculating a weighted sum of the multiple adaptive weights.
[0018] According to one embodiment of the present invention, the improved YOLOv5s algorithm replaces the loss function of the original YOLOv5s algorithm with Wise-IoU v3, and before acquiring the infrared image of the power transmission line to be detected, further includes:
[0019] Obtain a set of infrared images for training;
[0020] The infrared image set is labeled to obtain a labeled data set, wherein the labels are in YOLO format and are divided into six types of labels: insulator, suspension wire clamp, tension wire clamp, shock-absorbing hammer, grading ring and fault heating point;
[0021] Based on a preset ratio, the labeled data set is divided into a training set and a test set, and the improved YOLOv5s algorithm is trained using the training set, and after the training is completed, the improved YOLOv5s algorithm is tested using the test set based on Wise-IoU v3 to obtain a test result;
[0022] If the test result meets the preset test condition, the improved YOLOv5s algorithm is used to perform multi-target detection on the infrared image of the power transmission line to be detected; otherwise, the preset ratio is adjusted and retraining is performed based on the adjusted training set until the new test result meets the preset test condition.
[0023] According to the multi-target detection method of the infrared image of the power transmission line of the embodiment of the present invention, based on the Backbone part of the improved YOLOv5s algorithm, the attention weight feature information of the infrared image of the power transmission line to be detected is calculated, and the global feature relationship information is obtained according to the initial high-dimensional information and the attention weight feature information, and the global important information of the enhanced image is obtained according to the processed global feature relationship information and the initial high-dimensional information of the infrared image of the power transmission line to be detected; based on the Neck part of the improved YOLOv5s algorithm, the feature aggregation of the global important information of the enhanced image is performed to obtain the aggregated image information, and based on the Head part of the improved YOLOv5s algorithm, the target detection result is obtained according to the aggregated image information. In this way, the problems of low detection accuracy and performance degradation in practical applications of the traditional detection method are solved, and the accuracy and robustness of target detection are improved.
[0024] The second aspect of the present invention provides a multi-target detection device for infrared images of power transmission lines, wherein the device uses an improved YOLOv5s algorithm, wherein the improved YOLOv5s algorithm introduces a global context attention mechanism into the Backbone part of the original YOLOv5s algorithm, and adds multiple context enhancement modules into the Neck part of the original YOLOv5s algorithm, wherein the device comprises:
[0025] An acquisition module, used for acquiring an infrared image of a power transmission line to be detected;
[0026] A processing module, used to calculate the attention weight feature information of the infrared image of the power transmission line to be detected based on the Backbone part of the improved YOLOv5s algorithm, and obtain global feature relationship information according to the initial high-dimensional information of the infrared image of the power transmission line to be detected and the attention weight feature information, and process the global feature relationship information, and obtain enhanced global important information of the image according to the processed global feature relationship information and the initial high-dimensional information of the infrared image of the power transmission line to be detected;
[0027] The feature aggregation and detection module is used to perform feature aggregation on the global important information of the enhanced image based on the Neck part of the improved YOLOv5s algorithm to obtain aggregated image information, and to obtain target detection results based on the aggregated image information based on the Head part of the improved YOLOv5s algorithm.
[0028] According to one embodiment of the present invention, the target detection result includes at least one of the insulator position and insulator confidence, the suspension wire clamp position and suspension wire clamp confidence, the tension wire clamp position and tension wire clamp confidence, the shock-absorbing hammer position and shock-absorbing hammer confidence, the equalizing ring position and equalizing ring confidence, the fault hot spot position and the fault hot spot confidence.
[0029] According to an embodiment of the present invention, the feature aggregation and detection module is used to:
[0030] A detection frame is generated by using the Head part of the improved YOLOv5s algorithm, and classification, positioning and confidence scoring are performed based on the detection frame and the aggregated image information to obtain the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, anti-vibration hammer position, anti-vibration hammer confidence, grading ring position, grading ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected;
[0031] The target detection result is obtained according to the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, shock-absorbing hammer position, shock-absorbing hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected.
[0032] According to an embodiment of the present invention, the feature aggregation and detection module is used to:
[0033] Based on the multiple context enhancement modules, adaptively calculate the global important information of the enhanced image to obtain multiple adaptive weights;
[0034] The aggregated image information is obtained by calculating a weighted sum of the multiple adaptive weights.
[0035] According to one embodiment of the present invention, the improved YOLOv5s algorithm replaces the loss function of the original YOLOv5s algorithm with Wise-IoU v3. Before acquiring the infrared image of the power transmission line to be detected, the acquisition module is further used to:
[0036] Obtain a set of infrared images for training;
[0037] The infrared image set is labeled to obtain a labeled data set, wherein the labels are in YOLO format and are divided into six types of labels: insulator, suspension wire clamp, tension wire clamp, shock-absorbing hammer, grading ring and fault heating point;
[0038] Based on a preset ratio, the labeled data set is divided into a training set and a test set, and the improved YOLOv5s algorithm is trained using the training set, and after the training is completed, the improved YOLOv5s algorithm is tested using the test set based on Wise-IoU v3 to obtain a test result;
[0039] If the test result meets the preset test condition, the improved YOLOv5s algorithm is used to perform multi-target detection on the infrared image of the power transmission line to be detected; otherwise, the preset ratio is adjusted and retraining is performed based on the adjusted training set until the new test result meets the preset test condition.
[0040] According to the multi-target detection device of the infrared image of the power transmission line of the embodiment of the present invention, based on the Backbone part of the improved YOLOv5s algorithm, the attention weight feature information of the infrared image of the power transmission line to be detected is calculated, and the global feature relationship information is obtained according to the initial high-dimensional information and the attention weight feature information, and the global important information of the enhanced image is obtained according to the processed global feature relationship information and the initial high-dimensional information of the infrared image of the power transmission line to be detected; based on the Neck part of the improved YOLOv5s algorithm, the feature aggregation of the global important information of the enhanced image is performed to obtain the aggregated image information, and based on the Head part of the improved YOLOv5s algorithm, the target detection result is obtained according to the aggregated image information. In this way, the problem of low detection accuracy and performance degradation in practical applications of the traditional detection method is solved, and the accuracy and robustness of target detection are improved.
[0041] A third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-target detection method for infrared images of transmission lines as described in the above embodiments.
[0042] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the multi-target detection method for infrared images of power transmission lines as described in the above embodiments.
[0043] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0045] Figure 1 This is the structural diagram of the YOLOv5s network;
[0046] Figure 2 This is a schematic diagram of the GC block network structure;
[0047] Figure 3 This is a comparison diagram of the network structures of C3GC and C3;
[0048] Figure 4 A CAM (context enhancement) module structure diagram according to an embodiment of the present invention;
[0049] Figure 5 An improved YOLOv5s algorithm structure diagram according to an embodiment of the present invention;
[0050] Figure 6 A flowchart of a multi-target detection method for infrared images of a power transmission line provided according to an embodiment of the present invention;
[0051] Figure 7 Schematic diagram of Wise-IoU regression;
[0052] Figure 8 is a schematic diagram of an infrared grayscale image according to an embodiment of the present invention;
[0053] Fig. 9 A schematic diagram of a CAM fusion mechanism according to an embodiment of the present invention;
[0054] Fig.10 This is a comparison chart of mAP@0.5 in an ablation experiment according to an embodiment of the present invention;
[0055] Fig.11 A loss function effect diagram according to an embodiment of the present invention;
[0056] Fig.12 is a schematic diagram of an infrared image sample according to an embodiment of the present invention;
[0057] Fig.13 This is a diagram showing the prediction effect of the original YOLOv5s algorithm according to an embodiment of the present invention;
[0058] Fig.14 This is a diagram showing the prediction effect of an improved YOLOv5s algorithm according to an embodiment of the present invention;
[0059] Fig.15 2 is a block diagram of a multi-target detection device for infrared images of power transmission lines according to an embodiment of the present invention;
[0060] Fig.16 FIG. 4 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention.
[0061] Figure numerals: 10 - multi-target detection device for infrared images of power transmission lines, 100 - acquisition module, 200 - processing module, 300 - feature aggregation and detection module; 1601 - memory, 1602 - processor, 1603 - communication interface. DETAILED DESCRIPTION
[0062] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0063] The following describes a method, apparatus, device and medium for multi-target detection of infrared images of power transmission lines according to an embodiment of the present invention with reference to the accompanying drawings.
[0064] Before introducing the multi-target detection method for infrared images of transmission lines according to an embodiment of the present invention, the YOLOv5 algorithm, the YOLOv5s network structure, and the improved YOLOv5s algorithm involved in the multi-target detection method for infrared images of transmission lines proposed in the present invention are first introduced.
[0065] First, the YOLOv5 algorithm is introduced. The YOLOv5 algorithm is a popular single-stage target detection algorithm at this stage. It has made significant improvements in speed and lightweight design, and its model is smaller. It is more practical than the YOLOv7 and YOLOv8 algorithms that appeared later. Compared with the previous YOLOv4 and YOLOv3 algorithms, it improves detection accuracy and model performance while maintaining high speed. Among the YOLOv5 series algorithms, YOLOv5s has the smallest model and the fastest detection speed, which is suitable for processing tasks with high real-time requirements.
[0066] Secondly, the YOLOv5s network structure is introduced. The network structure of YOLOv5s is as follows Figure 1 As shown in (a), it mainly consists of three parts: Backbone, Neck, and Head. The Backbone part mainly consists of the following key parts: Convolutional layer (Conv), C3 module, and Spatial Pyramid Pooling Fast (SPPF) module. The Conv module is composed of the convolutional layer combined with the batch normalization layer and Silu activation function, which is used to extract target features. The network structure of the C3 module is shown in Figure 1 As shown in (b), the C3 module combines the Bottleneck structure for residual learning and Concat operations to enhance the feature learning capability and alleviate the gradient vanishing problem in deep networks. The network structure of the Bottleneck module is shown in Figure 1 As shown in (c), the main principle is to achieve dimensionality reduction and dimensionality increase of feature channels through the combination of 1x1 and 3x3 convolution layers, thereby reducing the computational complexity while maintaining the feature expression capability. Figure 1 As shown in (d), it improves the processing speed and optimizes the feature capture of multi-scale targets by using multiple small-sized pooling kernels. The Neck part adopts the Feature Pyramid Networks (FPN) structure, and strengthens the feature fusion and positioning information transmission through the supplement of the PANet structure. The Head part is responsible for generating detection boxes and classifying, locating and scoring them.
[0067] Finally, the improved YOLOv5s algorithm proposed in the present invention is introduced, wherein the improved YOLOv5s algorithm introduces a global contextual attention mechanism into the Backbone part of the original YOLOv5s algorithm, and adds multiple contextual enhancement modules into the Neck part of the original YOLOv5s algorithm.
[0068] Specifically, we first introduce the global contextual attention mechanism in the Backbone part of the original YOLOv5s algorithm.
[0069] The Global Context Block (GC) mechanism comes from the GC Net paper. The GC block is improved on the basis of the non-local network. Experiments have shown that capturing long-distance dependencies is beneficial to various recognition tasks. In traditional neural networks, long-distance dependencies can only be modeled by continuously stacking convolutional layers, which is inefficient and computationally intensive. For this reason, a better GC block is proposed.
[0070] Figure 2 The network structure is a GC block. The input image to be detected is first calculated through a 1×1 convolution block and a Softmax module to obtain the attention weight feature information of the image, and then the global feature relationship information C×1×1 is calculated with the input H×W×C, as shown in formula (1). Then, two 1×1 convolution blocks in the Transform structure are used to reduce the number of model parameters and save computational overhead. In addition, a LayerNorm module is inserted between the two 1×1 convolution blocks, which can improve the training stability of the model and has a certain regularization effect, reducing the risk of overfitting of the model. Finally, the initial information H×W×C is added to the global feature relationship information C×1×1 to obtain the output result of the enhanced global important information of the image, as shown in formula (3).
[0071]
[0072] Among them, α j represents the weight of global attention pooling, Represents the attention score in the attention mechanism, W k represents the weight matrix, x j Represents the input vector containing feature information of a certain position, N p is the total number of features, x m represents the input vector containing the global features after pooling or aggregation, m represents the dimension size (number of features) of the input feature vector, Represents the input vector x m After the weight matrix W k The weighted attention score, z krepresents the global context attention mechanism modeling architecture, F(·,·) represents the fusion function that aggregates the global context features into each position feature, and x k represents the input vector containing the intermediate feature results obtained in the calculation of global attention, δ(·) represents the feature transformation that captures channel dependencies, and x i Represents an input vector containing features at a specific position, ReLU() represents an activation function, LN() represents a normalization method for standardizing the input of each layer, and W v1 Represents the weight matrix, which performs a preliminary linear transformation on the features, W v2 Represents the weight matrix, which represents a further linear transformation of the features after normalization and activation function, z i Indicates that the input vector x i The total feature vector is obtained by fusing it with the transformed global context feature information.
[0073] Formula (1) represents the weight of global attention pooling, formula (2) represents the modeling architecture of global contextual attention mechanism, and formula (3) is the specific calculation formula of GC. Specifically, the calculation of GC mainly includes three parts. The first step is to calculate the weight α of global attention pooling. j ; The second step is to capture channel dependency through bottleneck transformation δ(·); The third step is to use the fusion function F(·,·) for feature fusion.
[0074] Further, Figure 3 The new module C3GC proposed in the present invention is compared with the existing C3 module in detail. In the Backbone part, as the number of network layers increases, the C3 module faces challenges in extracting shallow features, and is prone to the problem of gradient vanishing when the network depth increases. In view of this, the present invention cleverly embeds a GC block before one of the Conv modules, and the other Conv module remains unchanged. Compared with embedding GC blocks before both Conv modules, this improved method proposed in the present invention can introduce global context information according to specific needs while keeping the model calculation simple and effective. By choosing to fuse the GC block in one branch, the branch of the other GC block can continue to focus on learning fine-grained local features, thereby forming an effective complementarity between context information and detail information, and realizing the organic combination of global context and local features, thereby improving the performance of the model in target detection and image processing tasks.
[0075] It is understandable that, due to the presence of some small target devices when performing power transmission line detection, and the captured images are limited by the environment, shooting distance, shooting angle, etc., the quality of infrared images varies. Therefore, the present invention introduces the CAM (Context Augmentation Module) module in the Neck network part of the original YOLOv5s algorithm to balance the training process, reduce feature conflicts, improve the detection accuracy of small targets, and enhance the robustness of the model while maintaining computational efficiency. The CAM module expands the scale of the shared convolution kernel and increases the receptive field of the convolution kernel without changing the scale of the feature map.
[0076] The CAM module applies dilated convolutions with different dilated convolution rates to obtain contextual information of different receptive fields, and injects it into the FPN (Feature Pyramid Networks) network in YOLOv5 from top to bottom. Its structure is shown in the figure below. Figure 4 As shown in the figure, the features are processed by dilated convolution with 3×3 convolution kernels at different dilation ratios of 1, 3, and 5, so that the module can achieve the purpose of multi-scale extraction of feature maps.
[0077] Therefore, by introducing the global context attention mechanism into the Backbone part of the original YOLOv5s algorithm and entering the context enhancement module into the Neck part of the original YOLOv5s algorithm, the present invention proposes the following Figure 5 The improved YOLOv5s algorithm shown.
[0078] like Figure 5 As shown, Figure 5 The part circled by the dotted box is the module added by the present invention on the basis of the original YOLOv5s. First, in order to enhance the model's ability to extract global features and reduce computational overhead, the present invention embeds 4 C3GC modules in the Backbone part of the original YOLOv5s to extract features and improve the ability to extract global context features. Secondly, three CAM modules are added before the three concat modules in the Neck part of the original YOLOv5s. The CAM module is used to enhance the detection performance of small targets, and obtains context information of different receptive fields by applying dilated convolutions with different dilated convolution rates, thereby realizing multi-scale extraction of feature maps, and aggregating context information into the output feature map through three mechanisms of weighted fusion, cascade operation and adaptive operation, thereby reducing information conflicts and improving the fusion effect of feature maps.
[0079] Therefore, the improved YOLOv5s algorithm proposed in the present invention firstly introduces a global context attention mechanism in the Backbone part of the original YOLOv5s to enhance the model's ability to extract global features and reduce its computational overhead; secondly, a context enhancement module is introduced in the Neck part of the original YOLOv5s to improve the detection performance for small targets. The improved YOLOv5s algorithm proposed in the present invention solves the problems of low data set quality, dense equipment and different sizes in the operation detection of power transmission lines.
[0080] The following introduces the multi-target detection method of infrared images of power transmission lines using the above-mentioned improved YOLOv5s algorithm proposed by the present invention.
[0081] Specifically, Figure 6 A schematic flow chart of a multi-target detection method for infrared images of power transmission lines provided in an embodiment of the present invention.
[0082] like Figure 6 As shown, the multi-target detection method of the transmission line infrared image includes the following steps:
[0083] In step 601, an infrared image of a power transmission line to be inspected is acquired.
[0084] The infrared image of the power transmission line to be inspected refers to image data reflecting the surface temperature distribution of the power transmission line to be inspected and its ancillary equipment.
[0085] Specifically, the embodiments of the present invention can use an infrared thermal imager in the prior art to obtain an infrared image of the transmission line to be inspected. In addition, those skilled in the art can also use other methods in the prior art to obtain an infrared image of the transmission line to be inspected according to actual conditions, which is not specifically limited here.
[0086] Furthermore, in some embodiments, the improved YOLOv5s algorithm replaces the loss function of the original YOLOv5s algorithm with Wise-IoU v3, and before obtaining the infrared image of the transmission line to be detected, it also includes: obtaining an infrared image set for training; labeling the infrared image set to obtain a labeled data set, wherein the label adopts the YOLO format, and is divided into six labels: insulator, suspension wire clamp, tension wire clamp, shock-absorbing hammer, grading ring and fault heating point; based on a preset ratio, the labeled data set is divided into a training set and a test set, and the training set is used to train the improved YOLOv5s algorithm, and after the training is completed, the improved YOLOv5s algorithm is tested with the test set based on Wise-IoU v3 to obtain a test result; if the test result meets the preset test condition, the improved YOLOv5s algorithm is used to perform multi-target detection on the infrared image of the transmission line to be detected, otherwise, the preset ratio is adjusted, and retraining is performed based on the adjusted training set until the new test result meets the preset test condition.
[0087] It is understandable that target detection is crucial in computer vision, and its accuracy depends on the design of the loss function. The bounding box loss function is crucial to improving model performance. However, the target detection method of infrared images of power lines is usually trained and optimized based on high-quality data sets, while environmental factors in practice may cause the training data to contain a large number of ordinary and low-quality samples. If these samples are over-optimized, the model performance will be impaired. Therefore, the embodiment of the present invention replaces the loss function of YOLOv5s with Wise-IoU, which introduces a dynamic nonlinear focusing mechanism, evaluates the anchor frame through "outlier degree", intelligently allocates gradient gain, reduces dependence on low-quality samples, and improves detector performance. There are three versions of Wise-IoU, namely v1, v2, and v3. Wise-IoU v1 adopts a two-level attention mechanism based on distance, and its calculation formula is:
[0088]
[0089] in, is the bounding box loss under the Wise-IoU v1 loss function, is the bounding box loss under IoU, R WloU is the distance attention, (x,y) is the predicted box of the target, (x gt ,y gt ) is the real box of the target, W g With H g are the width and height of the minimum bounding rectangle of the predicted box and the real box respectively. The superscript * indicates that W g With H g Separating from the computational graph can effectively improve the convergence efficiency. The regression diagram is shown in Figure 7 .
[0090] Compared with Wise-IoU v1, Wise-IoU v3 adopted in the embodiment of the present invention does not have the calculation related to the aspect ratio and distance attention, but adopts a dynamic non-monotonic focusing mechanism and introduces "outlier" to describe the quality of the anchor frame. When the outlier of the anchor frame is small, it means that its quality is high, and the model will assign a small gradient gain to it. At the same time, when the outlier of the anchor frame is particularly large (the quality of the anchor frame is particularly low), the model also assigns a small gradient gain to it to prevent it from generating large harmful gradients, so that the model can focus more on ordinary and lower quality anchor frames, thereby improving the detection performance of the model under complex backgrounds. The calculation formula of Wise-IoU v3 is as follows:
[0091]
[0092] Wherein, r in formula (6) is the non-monotonic focusing parameter, β is the outlier degree, δ and α are hyperparameters, and in the present invention, the hyperparameters δ and α are set to 3 and 1.9 respectively. is the sliding average, is the gradient gain, and in formula (8) is the bounding box loss under the Wise-IoU v3 loss function. And, since is updated in real time, which enables Wise-IoU v3 to dynamically adjust the gradient gain according to the current training status.
[0093] Therefore, by replacing the loss function with Wise-IoU v3 and adopting a dynamic non-monotonic focusing mechanism, the model is made to focus more on common and lower-quality anchor boxes, which improves the detection performance of the model and makes the model more practical.
[0094] In summary, the improved YOLOv5s algorithm in the embodiment of the present invention adopts Wise-IoU v3 to replace the loss function in YOLOv5s, because Wise-IoU v3 not only takes into account the overlapping area of the target frame, but also focuses on the anchor frame of ordinary quality, so it can better handle the situation of inaccurate target boundary. This method can effectively improve the target detection accuracy of the model under complex backgrounds, especially in scenes with many small targets and obstructions, thereby improving the positioning performance of the present invention for multiple devices in the power transmission line.
[0095] Furthermore, the improved YOLOv5s algorithm is trained and tested, and when the test results meet the preset test conditions, the improved YOLOv5s algorithm is used to perform multi-target detection on the infrared image of the transmission line to be inspected.
[0096] Specifically, firstly, infrared thermal imaging technology is used to obtain an infrared image set for training, and the infrared images in the infrared image set are grayed. Figure 8 As shown in the figure, a labeling tool (such as LabelImg tool) is used to label the infrared image set after grayscale processing to obtain a labeled data set. The labels are in YOLO format and are divided into six types of labels: insulator, suspension clamp, tension clamp, shock-absorbing hammer, grading ring and fault heating point.
[0097] Furthermore, the labeled data set is divided into a training set and a test set according to a preset ratio. For example, taking the number of infrared images in the infrared image set as 1270 and the preset ratio as 7:3, the divided training set, test set and the number of various labels are shown in Table 1.
[0098] Table 1
[0099] name quantity Training set 889 Test Set 381 insulator 1845 X clamp 482 N clamp 249 hammer 1168 ring 280 error 253
[0100] In Table 1, insulator represents the insulator in the infrared image, X clamp represents the suspension clamp in the infrared image, N clamp represents the tension clamp in the infrared image, hammer represents the anti-vibration hammer in the infrared image, ring represents the equalizing ring in the infrared image, and error represents the possible fault hot spot in the infrared image.
[0101] Furthermore, the improved YOLOv5s algorithm is used as the basic model, and the model is trained using the training set. The training environment of the embodiment of the present invention is under the Windows 10 operating system, using NVIDIA GeForce GTX 1650 as hardware, Pytorch as the deep learning framework, Python version 3.8, CUDA version 12.1, and the training epoch is set to 100. Among them, the momentum is 0.937, the weight decay is 0.0005, the batch size is 8, the learning rate is 0.0001, and the input image size is 640×640.
[0102] Furthermore, after the model training is completed, each image in the test set is traversed, and for each image, its true label is obtained. For each predicted box in each image, the intersection-over-union ratio between it and the true box is calculated using Wise-IoU v3. If the intersection-over-union ratio exceeds a certain threshold (for example, 0.8), the prediction is determined to be a correct match.
[0103] Furthermore, the number of correct matches for each category was counted, and the performance of the model on the test set was evaluated using the precision, recall, and mean average precision (mAP) evaluation indicators.
[0104] Among them, precision refers to the proportion of positive samples correctly predicted by the model. The calculation formula is shown in formula (9).
[0105]
[0106] Among them, P is the precision, T P is the number of samples correctly predicted as positive, F P is the number of samples that are incorrectly predicted to be positive.
[0107] Recall rate refers to the proportion of positive samples predicted by the model to all positive samples. The calculation formula is shown in formula (10), F N It is expressed as the number of samples that the model incorrectly predicts as negative.
[0108]
[0109] Among them, R is the recall rate, T P is the number of samples correctly predicted as positive, F N is the number of samples that the model incorrectly predicts as negative.
[0110] The mean average precision is obtained by averaging the average precision (AP) of a single category, and the calculation formula is shown in formula (11). AP is determined by calculating the area under the PR curve, and the calculation formula is shown in formula (12).
[0111]
[0112] Among them, L mAP is the average precision, L AP is the average precision, n is the number of segments on the PR curve, that is, the number of data points for calculating the average precision, and P(R) is the precision-recall (PR) curve function.
[0113] Furthermore, the test results are obtained based on the precision, recall and average precision. If the test results meet the preset test conditions, such as when the precision, recall and average precision mean reaches a specific threshold, the test results are judged to meet the preset test conditions, that is, the model training is successful, and the improved YOLOv5s algorithm is used to perform multi-target detection on the infrared image of the power transmission line to be detected. If the test results do not meet the preset test conditions, it means that the performance of the model does not meet expectations, and it is necessary to adjust the division ratio of the data set and retrain based on the adjusted training set until the new test results meet the preset test conditions.
[0114] It should be noted that, after each round of training, the embodiment of the present invention uses Wise-IoU v3 to calculate the loss result, and feeds the loss result back to the improved YOLOv5s model to update the parameters of the loss function, and performs the next round of training after the parameters of the loss function are updated.
[0115] In step S602, based on the Backbone part of the improved YOLOv5s algorithm, the attention weight feature information of the infrared image of the transmission line to be detected is calculated, and the global feature relationship information is obtained according to the initial high-dimensional information of the infrared image of the transmission line to be detected and the attention weight feature information, and the global feature relationship information is processed, and the enhanced global important information of the image is obtained according to the processed global feature relationship information and the initial high-dimensional information of the infrared image of the transmission line to be detected.
[0116] Specifically, the infrared image of the transmission line to be detected is input into the Backbone part of the improved YOLOv5s algorithm. Figure 2 The 1×1 convolution block and the Softmax module in the GC block shown calculate the attention weight feature information of the infrared image of the transmission line to be detected, and calculate the C×1×1 global feature relationship information based on the initial high-dimensional information H×W×C of the input infrared image of the transmission line to be detected.
[0117] Furthermore, if Figure 2 As shown in the figure, two 1×1 convolution blocks in the Transform structure are used to reduce the number of model parameters and save computational overhead. A LayerNorm module is inserted between the two 1×1 convolution blocks, which can improve the training stability of the model and has a certain regularization effect, reducing the risk of model overfitting. Finally, the initial high-dimensional information H×W×C of the infrared image of the power transmission line to be detected is added to the global feature relationship information C×1×1 to obtain the output result of the enhanced global important information of the image.
[0118] In step S603, based on the Neck part of the improved YOLOv5s algorithm, feature aggregation is performed on the important global information of the enhanced image to obtain aggregated image information, and based on the Head part of the improved YOLOv5s algorithm, the target detection result is obtained according to the aggregated image information.
[0119] Among them, in some embodiments, the target detection results include at least one of the insulator position and insulator confidence, the suspension wire clamp position and suspension wire clamp confidence, the tension wire clamp position and tension wire clamp confidence, the shock-absorbing hammer position and shock-absorbing hammer confidence, the equalizing ring position and equalizing ring confidence, the fault hot spot position and the fault hot spot confidence.
[0120] Furthermore, in some embodiments, based on the Neck part of the improved YOLOv5s algorithm, feature aggregation is performed on the global important information of the enhanced image to obtain aggregated image information, including: based on multiple context enhancement modules, adaptively calculating the global important information of the enhanced image to obtain multiple adaptive weights; and calculating the weighted sum of the multiple adaptive weights to obtain the aggregated image information.
[0121] The context enhancement module of the embodiment of the present invention includes three fusion mechanisms, such as Fig. 9 As shown, they are weighted fusion, cascade operation and adaptive operation respectively. Fig. 9 (a) and Fig. 9 The fusion mechanism shown in (b) is to adjust the number of passes of the feature map through three 1×1 convolutions, and then perform feature fusion in the spatial and channel dimensions. Fig. 9 The adaptive operation represented by (c) is convolution, concatenation and Softmax function to obtain adaptive weights. By calculating the weighted sum, the context information can be aggregated to the output. The connection mode of the context enhancement module of the embodiment of the present invention is as follows: Fig. 9 (c) shows the Adaptive connection mode.
[0122] Specifically, the global important information of the enhanced image is input into the Neck part of the improved YOLOv5s algorithm, the number of channels of the enhanced image is adjusted through the convolution layer (for example, 3 1×1 convolutions), and feature fusion is performed in the spatial and channel dimensions. The global important information of the enhanced image is adaptively calculated through convolution, splicing and Softmax function. For each context enhancement module, the enhanced image is weighted and summed according to the adaptive weight of its output to obtain the aggregated output of the context enhancement module. The aggregated results of all context enhancement modules are weighted and summed to obtain the final aggregated image information.
[0123] Further, in some embodiments, based on the Head part of the improved YOLOv5s algorithm, the target detection result is obtained according to the aggregated image information, including: generating a detection frame using the Head part of the improved YOLOv5s algorithm, and performing classification, positioning and confidence scoring based on the detection frame and the aggregated image information to obtain the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, anti-shock hammer position, anti-shock hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected; obtaining the target detection result according to the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, anti-shock hammer position, anti-shock hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected.
[0124] Specifically, the aggregated image information is input into the Head part of the improved YOLOv5s algorithm, and the Head part of the improved YOLOv5s algorithm is used to generate a detection frame. Based on the detection frame and the aggregated image information, the category to which each detection frame belongs (such as insulator, suspension wire clamp, tension wire clamp, etc.), the center coordinates of each detection frame, the aspect ratio and the confidence score of each detection frame belonging to a certain category are predicted.
[0125] Furthermore, for each detected insulator, its position (coordinates) and confidence score in the infrared image of the transmission line to be detected are output. Similarly, for the detected suspension wire clamp, tension wire clamp, shock-absorbing hammer equalizing ring and fault hot spot position, the suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, shock-absorbing hammer position, shock-absorbing hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence are output respectively.
[0126] Finally, the target detection result is obtained according to the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, shock-absorbing hammer position, shock-absorbing hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected.
[0127] In addition, in order to verify the effectiveness of the improvements such as adding C3GC module, adding CAM module, and changing Wise-IoU loss function to the original YOLOv5s algorithm, this paper conducted ablation experiments and compared the results. Fig.10 As shown, Fig.10 The horizontal axis represents the number of training rounds, and the vertical axis represents mAP@0.5. Fig.10In the figure, the blue curve YOLOv5s represents the original YOLOv5s algorithm, the orange curve YOLOv5s+Wise-iou represents the improved algorithm after replacing Wise-IoU v3, the green curve YOLOv5s+CAM represents the improved algorithm after adding the CAM module, the red curve YOLOv5s+C3GC represents the improved algorithm after adding the C3GC module, and the purple curve YOLOv5s+ECW represents the improved YOLOv5s algorithm after integrating the above three improvement methods.
[0128] Depend on Fig.10 It can be seen that compared with the original YOLOv5s algorithm, the improved YOLOv5s algorithm has not only improved in accuracy, but also the curve converges faster than the original YOLOv5s curve. The curve of YOLOv5s+ECW (improved YOLOv5s algorithm) is smoother after reaching the peak accuracy.
[0129] The specific data are shown in Table 2. Model 1, Model 2, and Model 3 have increased mAP@0.5% by 4.0%, 2.6%, and 1.8% compared to the original YOLOv5s algorithm. Model 4, which is the final improved YOLOv5s algorithm after integrating the three improvements, has a mAP@0.5% higher than the original YOLOv5s algorithm by 4.8%, and the precision and recall rates have been improved to varying degrees, proving the effectiveness and feasibility of the improved YOLOv5s algorithm.
[0130] Table 2
[0131] Model YOLOv5s +C3GC +CAM +Wise-IoU P / % R / % mAP@0.5 / % 0 √ 76.5 67.0 69.3 1 √ √ 77.6 71.3 72.6 2 √ √ 80.7 68.1 71.2 3 √ √ 76.0 68.7 70.4 4 √ √ √ √ 84.5 71.5 73.2
[0132] Further, Fig.11 This is a comparison chart of the loss function effects of the original YOLOv5s algorithm and the improved YOLOv5s algorithm proposed in this invention. Fig.11 (a) is the image of the training classification loss function. Fig.11 (b) is the image for verifying the classification loss function. Fig.11 The orange curve in the figure is the loss function of the original YOLOv5s algorithm, and the blue curve is the loss function of the improved YOLOv5s algorithm. Fig.11 It can be seen that the improved YOLOv5s algorithm has a smaller loss value than the original YOLOv5s, and the curve tends to stabilize faster, especially on the test set, which reflects the effectiveness of replacing the Wise-IoU v3 loss function in the present invention.
[0133] Furthermore, the present invention selects four images from the infrared images to test the detection effects of the original YOLOv5s algorithm and the improved YOLOv5s algorithm. Fig.12 , Fig.13 and Fig.14 As shown, Fig.12 is a sample infrared image. Fig.12 Including four infrared image samples (a), (b), (c) and (d). Fig.13 It uses the YOLOv5s algorithm to Fig.12 (a) Fig.12 (b) Fig.12 (c) and Fig.12 (d) The effect diagram after prediction, Fig.14 It uses the improved YOLOv5s algorithm to Fig.12 (a) Fig.12 (b) Fig.12 (c) and Fig.12 (d) The predicted effect diagram. Fig.13 and Fig.14 The component category and confidence level are displayed on the rectangular box for component positioning. Insulator represents insulator, X clamp represents suspension clamp, N clamp represents tension clamp, hammer represents shock-absorbing hammer, ring represents equalizing ring, and error represents possible fault hot spots in the infrared image.
[0134] When the original YOLOv5s algorithm detects images, some parts of the power equipment are not detected due to the dark background and the small size of the equipment. In addition, the confidence of the rectangular box of the original YOLOv5s algorithm for detecting smaller targets such as shock-absorbing hammers, tension clamps, and suspension clamps is not high. This is because before the original YOLOv5s was improved, when performing multi-target detection of transmission lines, it tended to identify large and numerous targets such as insulators, and ignored small targets such as shock-absorbing hammers and tension clamps, resulting in low detection accuracy for small targets. However, the improved YOLOv5s algorithm proposed in the present invention avoids this problem. Comparison Fig.12 and Fig.13 It can be seen that Fig.12 In the leftmost image (a) of the four images, the original YOLOv5s algorithm missed a shock absorber when detecting it. Fig.12 In (b), the original YOLOv5s algorithm identifies the two anti-vibration hammers and the two tension clamps as one during detection. Therefore, the original YOLOv5s algorithm has a serious problem of missing small targets.
[0135] The improved YOLOv5s algorithm proposed in the present invention focuses more on small targets, which effectively avoids the occurrence of missed detection problems. The detection of small targets such as anti-vibration hammers and tension clamps is more accurate and has higher confidence than the original YOLOv5s algorithm. In addition, the method proposed in the present invention adopts a contextual attention module and a multi-scale fusion mechanism, so it is obviously better than the original YOLOv5s algorithm for feature extraction and aggregation. In other words, the improved YOLOv5s algorithm performs better on low-quality data sets. The reason is that the quality of infrared images taken in practice is uneven due to the limitations of the environment, brightness, and distance, so the algorithm needs more ability to extract and aggregate features, which intuitively proves the effectiveness of the method proposed in the present invention in the field of power transmission line image detection.
[0136] Furthermore, Table 3 shows the recognition of various targets by the original YOLOv5s algorithm and the improved YOLOv5s algorithm of the present invention. The improved YOLOv5s algorithm proposed in the present invention does not significantly improve the recognition effect of insulators, but for small targets that are difficult to detect, such as tension clamps and suspension clamps, the detection accuracy of the improved YOLOv5s algorithm for tension clamps, suspension clamps, and shock-absorbing hammers is improved by 5.6%, 7.9%, and 3.3%, respectively, compared with the original YOLOv5s algorithm, further verifying the effectiveness of the improved YOLOv5s algorithm.
[0137] Table 3
[0138]
[0139] Further, in order to verify the superiority of the improved YOLOv5s algorithm of the present invention, the data set of the present invention is compared with a variety of mainstream detection algorithms, as shown in Table 4. It can be seen that the performance of the original YOLOv5s algorithm is improved to varying degrees relative to the SSD and YOLOv7 algorithms, but compared with algorithms such as Faster R-CNN and YOLOv8, although the accuracy of the two algorithms is similar to that of the original YOLOv5s algorithm, the original YOLOv5s algorithm is smaller than Faster R-CNN and YOLOv8, so the original YOLOv5s algorithm requires less computing resources during deployment and operation, so the comprehensive comparison is still the original YOLOv5s algorithm is more suitable for industrial practical fields, so the present invention adopts the original YOLOv5s algorithm and improves it on this basis. The data proves that the improved YOLOv5s algorithm (YOLOv5s-ECW in Table 4) has improved indicators such as precision and recall compared to before the improvement, and can well meet the requirements of infrared image detection of power transmission lines.
[0140] Table 4
[0141]
[0142]
[0143] Therefore, the present invention proposes an improved YOLOv5s algorithm based on the original YOLOv5s for multi-target detection of infrared images of power transmission lines. First, the C3GC module is introduced in the Backbone part to enhance the model's ability to extract global context features. Secondly, the CAM context enhancement module is introduced in the Neck part, and finally the new loss function Wise-IoU is replaced. The test results show that the average accuracy of the improved YOLOv5s algorithm is improved by 4.8% compared with the original YOLOv5s algorithm. At the same time, the precision rate, recall rate, etc. are improved to varying degrees. Moreover, the improved YOLOv5s algorithm can better focus on the detection of small targets and can better meet the requirements of practicality than the original YOLOv5s.
[0144] According to the multi-target detection method of the infrared image of the transmission line of the embodiment of the present invention, based on the Backbone part of the improved YOLOv5s algorithm, the attention weight feature information of the infrared image of the transmission line to be detected is calculated, and the global feature relationship information is obtained according to the initial high-dimensional information and the attention weight feature information, and the global important information of the enhanced image is obtained according to the processed global feature relationship information and the initial high-dimensional information of the infrared image of the transmission line to be detected; based on the Neck part of the improved YOLOv5s algorithm, the feature aggregation of the global important information of the enhanced image is performed to obtain the aggregated image information, and based on the Head part of the improved YOLOv5s algorithm, the target detection result is obtained according to the aggregated image information. In this way, the problem that the quality of the infrared image data set is not high due to factors such as the environment, shooting angle, and shooting distance in the infrared image detection of the transmission line, and the recognition accuracy is low due to the imbalance of the number of recognized targets and the different target sizes, etc., is solved, and the reliability and accuracy of the infrared image detection of the transmission line are improved.
[0145] Next, the multi-target detection device for infrared images of power transmission lines according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[0146] Fig.15 It is a block diagram of a multi-target detection device for infrared images of power transmission lines according to an embodiment of the present invention.
[0147] In this embodiment, the device uses an improved YOLOv5s algorithm, which introduces a global context attention mechanism into the Backbone part of the original YOLOv5s algorithm and adds multiple context enhancement modules into the Neck part of the original YOLOv5s algorithm.
[0148] like Fig.15As shown, the multi-target detection device 10 for infrared images of power transmission lines includes: an acquisition module 100 , a processing module 200 and a feature aggregation and detection module 300 .
[0149] Among them, the acquisition module 100 is used to acquire the infrared image of the transmission line to be detected; the processing module 200 is used to calculate the attention weight feature information of the infrared image of the transmission line to be detected based on the Backbone part of the improved YOLOv5s algorithm, and obtain the global feature relationship information according to the initial high-dimensional information of the infrared image of the transmission line to be detected and the attention weight feature information, and process the global feature relationship information, and obtain the global important information of the enhanced image according to the processed global feature relationship information and the initial high-dimensional information of the infrared image of the transmission line to be detected; the feature aggregation and detection module 300 is used to perform feature aggregation on the global important information of the enhanced image based on the Neck part of the improved YOLOv5s algorithm to obtain aggregated image information, and obtain the target detection result according to the aggregated image information based on the Head part of the improved YOLOv5s algorithm.
[0150] Further, in some embodiments, the target detection results include at least one of the insulator position and insulator confidence, the suspension wire clamp position and suspension wire clamp confidence, the tension wire clamp position and tension wire clamp confidence, the shock-absorbing hammer position and shock-absorbing hammer confidence, the equalizing ring position and equalizing ring confidence, the fault hot spot position and the fault hot spot confidence.
[0151] Further, in some embodiments, the feature aggregation and detection module 300 is used to: generate a detection frame using the Head part of the improved YOLOv5s algorithm, and perform classification, positioning and confidence scoring based on the detection frame and the aggregated image information to obtain the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, shockproof hammer position, shockproof hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected; obtain the target detection result according to the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, shockproof hammer position, shockproof hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected.
[0152] Furthermore, in some embodiments, the feature aggregation and detection module 300 is used to: based on multiple context enhancement modules, perform adaptive operations on the globally important information of the enhanced image to obtain multiple adaptive weights; and perform weighted calculations on the multiple adaptive weights to obtain aggregated image information.
[0153] Further, in some embodiments, the improved YOLOv5s algorithm replaces the loss function of the original YOLOv5s algorithm with Wise-IoU v3. Before obtaining the infrared image of the transmission line to be detected, the acquisition module 100 is also used to: obtain an infrared image set for training; label the infrared image set to obtain a labeled data set, wherein the label adopts the YOLO format, and is divided into six labels: insulator, suspension wire clamp, tension wire clamp, shock-absorbing hammer, grading ring and fault heating point; based on a preset ratio, the labeled data set is divided into a training set and a test set, and the training set is used to train the improved YOLOv5s algorithm, and after the training is completed, the improved YOLOv5s algorithm is tested with the test set based on Wise-IoU v3 to obtain a test result; if the test result meets the preset test condition, the improved YOLOv5s algorithm is used to perform multi-target detection on the infrared image of the transmission line to be detected, otherwise, the preset ratio is adjusted, and retraining is performed based on the adjusted training set until the new test result meets the preset test condition.
[0154] It should be noted that the above explanation of the embodiment of the multi-target detection method for infrared images of power transmission lines is also applicable to the multi-target detection device for infrared images of power transmission lines of this embodiment, and will not be repeated here.
[0155] According to the multi-target detection device of the infrared image of the power transmission line of the embodiment of the present invention, based on the Backbone part of the improved YOLOv5s algorithm, the attention weight feature information of the infrared image of the power transmission line to be detected is calculated, and the global feature relationship information is obtained according to the initial high-dimensional information and the attention weight feature information, and the global important information of the enhanced image is obtained according to the processed global feature relationship information and the initial high-dimensional information of the infrared image of the power transmission line to be detected; based on the Neck part of the improved YOLOv5s algorithm, the feature aggregation of the global important information of the enhanced image is performed to obtain the aggregated image information, and based on the Head part of the improved YOLOv5s algorithm, the target detection result is obtained according to the aggregated image information. In this way, the problem of low detection accuracy and performance degradation in practical applications of the traditional detection method is solved, and the accuracy and robustness of target detection are improved.
[0156] Fig.16 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:
[0157] A memory 1601 , a processor 1602 , and a computer program stored in the memory 1601 and executable on the processor 1602 .
[0158] When the processor 1602 executes the program, the multi-target detection method for the infrared image of the power transmission line provided in the above embodiment is implemented.
[0159] Furthermore, the electronic device further comprises:
[0160] The communication interface 1603 is used for communication between the memory 1601 and the processor 1602 .
[0161] The memory 1601 is used to store computer programs that can be executed on the processor 1602 .
[0162] The memory 1601 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0163] If the memory 1601, the processor 1602 and the communication interface 1603 are implemented independently, the communication interface 1603, the memory 1601 and the processor 1602 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.16 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0164] Optionally, in a specific implementation, if the memory 1601, the processor 1602 and the communication interface 1603 are integrated on a chip, the memory 1601, the processor 1602 and the communication interface 1603 can communicate with each other through an internal interface.
[0165] The processor 1602 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.
[0166] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned multi-target detection method for infrared images of power transmission lines.
[0167] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0168] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0169] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A multi-target detection method for infrared images of power transmission lines, characterized in that: The multi-target detection method of the infrared image of the power transmission line utilizes an improved YOLOv5s algorithm, wherein the improved YOLOv5s algorithm introduces a global context attention mechanism into the Backbone part of the original YOLOv5s algorithm, and adds multiple context enhancement modules into the Neck part of the original YOLOv5s algorithm, wherein the method comprises the following steps: Acquire infrared images of the power transmission lines to be inspected; Based on the Backbone part of the improved YOLOv5s algorithm, the attention weight feature information of the infrared image of the power transmission line to be detected is calculated, and the global feature relationship information is obtained according to the initial high-dimensional information of the infrared image of the power transmission line to be detected and the attention weight feature information, and the global feature relationship information is processed, and the global important information of the enhanced image is obtained according to the processed global feature relationship information and the initial high-dimensional information of the infrared image of the power transmission line to be detected; Based on the Neck part of the improved YOLOv5s algorithm, feature aggregation is performed on the global important information of the enhanced image to obtain aggregated image information, and based on the Head part of the improved YOLOv5s algorithm, a target detection result is obtained according to the aggregated image information.
2. The multi-target detection method of power transmission line infrared image according to claim 1 is characterized in that: The target detection result includes at least one of the insulator position and insulator confidence, the suspension wire clamp position and suspension wire clamp confidence, the tension wire clamp position and tension wire clamp confidence, the shock-absorbing hammer position and shock-absorbing hammer confidence, the equalizing ring position and equalizing ring confidence, the fault hot spot position and the fault hot spot confidence.
3. The multi-target detection method of power transmission line infrared image according to claim 2 is characterized in that: The Head part based on the improved YOLOv5s algorithm obtains the target detection result according to the aggregated image information, including: A detection frame is generated by using the Head part of the improved YOLOv5s algorithm, and classification, positioning and confidence scoring are performed based on the detection frame and the aggregated image information to obtain the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, anti-vibration hammer position, anti-vibration hammer confidence, grading ring position, grading ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected; The target detection result is obtained according to the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, shock-absorbing hammer position, shock-absorbing hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected.
4. The multi-target detection method of power transmission line infrared image according to claim 1 is characterized in that: Based on the Neck part of the improved YOLOv5s algorithm, feature aggregation is performed on the important global information of the enhanced image to obtain aggregated image information, including: Based on the multiple context enhancement modules, adaptively calculate the global important information of the enhanced image to obtain multiple adaptive weights; The aggregated image information is obtained by calculating a weighted sum of the multiple adaptive weights.
5. The multi-target detection method of power transmission line infrared image according to claim 1, characterized in that: The improved YOLOv5s algorithm replaces the loss function of the original YOLOv5s algorithm with Wise-IoU v3, and before acquiring the infrared image of the power transmission line to be detected, further includes: Obtain a set of infrared images for training; The infrared image set is labeled to obtain a labeled data set, wherein the labels are in YOLO format and are divided into six types of labels: insulator, suspension wire clamp, tension wire clamp, shock-absorbing hammer, grading ring and fault heating point; Based on a preset ratio, the labeled data set is divided into a training set and a test set, and the improved YOLOv5s algorithm is trained using the training set, and after the training is completed, the improved YOLOv5s algorithm is tested using the test set based on Wise-IoU v3 to obtain a test result; If the test result meets the preset test condition, the improved YOLOv5s algorithm is used to perform multi-target detection on the infrared image of the power transmission line to be detected; otherwise, the preset ratio is adjusted and retraining is performed based on the adjusted training set until the new test result meets the preset test condition.
6. A multi-target detection device for infrared images of power transmission lines, characterized in that: The multi-target detection device for infrared images of power transmission lines utilizes an improved YOLOv5s algorithm, wherein the improved YOLOv5s algorithm introduces a global context attention mechanism into the Backbone part of the original YOLOv5s algorithm, and adds multiple context enhancement modules into the Neck part of the original YOLOv5s algorithm, wherein the device comprises: An acquisition module, used for acquiring an infrared image of a power transmission line to be detected; A processing module, used to calculate the attention weight feature information of the infrared image of the power transmission line to be detected based on the Backbone part of the improved YOLOv5s algorithm, and obtain global feature relationship information according to the initial high-dimensional information of the infrared image of the power transmission line to be detected and the attention weight feature information, and process the global feature relationship information, and obtain enhanced global important information of the image according to the processed global feature relationship information and the initial high-dimensional information of the infrared image of the power transmission line to be detected; The feature aggregation and detection module is used to perform feature aggregation on the global important information of the enhanced image based on the Neck part of the improved YOLOv5s algorithm to obtain aggregated image information, and to obtain target detection results based on the aggregated image information based on the Head part of the improved YOLOv5s algorithm.
7. The multi-target detection device for infrared images of power transmission lines according to claim 6, characterized in that: The target detection result includes at least one of the insulator position and insulator confidence, the suspension wire clamp position and suspension wire clamp confidence, the tension wire clamp position and tension wire clamp confidence, the shock-absorbing hammer position and shock-absorbing hammer confidence, the equalizing ring position and equalizing ring confidence, the fault hot spot position and the fault hot spot confidence.
8. The multi-target detection device for infrared images of power transmission lines according to claim 7, characterized in that: The feature aggregation and detection module is used to: The Head part of the improved YOLOv5s algorithm is used to generate a detection frame, and classification, positioning and confidence scoring are performed based on the detection frame and the aggregated image information to obtain the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, anti-vibration hammer position, anti-vibration hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected; The target detection result is obtained according to the insulator position, insulator confidence, suspension wire clamp position, suspension wire clamp confidence, tension wire clamp position, tension wire clamp confidence, shock-absorbing hammer position, shock-absorbing hammer confidence, equalizing ring position, equalizing ring confidence, fault hot spot position and fault hot spot confidence in the infrared image of the transmission line to be detected.
9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-target detection method for infrared images of power transmission lines as described in any one of claims 1 to 5.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the multi-target detection method for infrared images of power transmission lines as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Distribution line tree obstacle detection method
CN115205292A
Aerial image rotating target detection method based on annular smooth label
CN116597324A
High-altitude parabolic object detection method and system based on deep learning, medium and equipment
CN117079095A
Improved multi-target detection method based on YOLOv5s network model
CN117789002A
Sparse convolution power transmission line external damage prevention detection method based on global context enhancement
CN117830959A
