Infrared defect discrimination method based on improved neural network and related device

Abstract

The invention provides an infrared defect discrimination method based on an improved neural network and a related device, and the method comprises the steps: obtaining an infrared image of power inspection, and carrying out the processing through employing a constructed infrared equipment image recognition model and a temperature difference recognition model, and obtaining a target detection and temperature difference result. The temperature difference result reflects the difference between the target current temperature and the reference temperature. And based on the temperature difference judgment basis of the equipment defect, carrying out logic analysis on the detection result to determine the equipment defect type. An infrared equipment image recognition model is based on an improved YOLOv5 network, and the target recognition effect is improved by introducing an Octave Conv module and a CARAFE operator to optimize feature extraction and up-sampling stages. The improved recognition model is used for automatically processing the infrared image, manual interpretation is reduced, misjudgment caused by visual fatigue is avoided, the method is not limited by professional ability of personnel, the judgment efficiency is improved, the processing time is shortened, and the infrared heating defect problem can be solved in time.

Description

Technical Field

[0001] This invention belongs to the field of infrared defect detection technology for power equipment, specifically relating to an infrared defect discrimination method and related device based on an improved neural network. Background Technology

[0002] In the power system sector, ensuring the safe and stable operation of power equipment is of paramount importance. With the continuous increase in the number of power devices and the growing complexity of operating environments, the requirements for power equipment condition monitoring technologies are also becoming increasingly stringent. Among these, using infrared imaging for thermal defect detection of power equipment has become an important non-contact detection method, capable of promptly identifying potential thermal faults and preventing accidents.

[0003] In recent years, due to the complexity and massive data volume of infrared images of power equipment, the automatic identification of infrared images of electrical equipment using computer vision technology has received widespread attention. Related technologies are constantly developing, such as infrared image recognition methods for power equipment and visual recognition-based methods for detecting wooden boards.

[0004] However, traditional infrared defect identification methods mostly rely on on-site inspections by maintenance personnel or the deployment of drones to capture images, followed by manual processing of large amounts of image information. Since personnel are prone to visual fatigue, this easily leads to missed or incorrect identifications. Furthermore, the effectiveness of the analysis is affected by the professional competence of the analysts, making the entire identification process inefficient and time-consuming, potentially resulting in infrared heating defects not being addressed in a timely manner. Summary of the Invention

[0005] In view of this, the present invention aims to provide an infrared defect discrimination method and related device based on an improved neural network, so as to improve the analysis efficiency of infrared defect type judgment of power equipment in complex substation scenarios, reduce manual analysis costs, improve analysis accuracy, and enable the system to adapt to the identification of power equipment in complex substation environments.

[0006] To achieve the above objectives, the technical solution provided by the present invention is as follows:

[0007] In a first aspect, the present invention provides an infrared defect discrimination method based on an improved neural network, comprising the following steps:

[0008] Acquire infrared images during power line inspection;

[0009] For infrared images, the pre-built infrared device image recognition model and infrared temperature difference recognition model are used for recognition to obtain target detection results and temperature difference detection results for each target; the temperature difference detection result is the temperature difference between the current temperature of the target and the preset reference temperature;

[0010] Based on the temperature difference judgment criteria for different equipment defects, logical judgment is performed on the target detection results and the corresponding temperature difference detection results to obtain the equipment defect type judgment result;

[0011] Among them, the infrared device image recognition model is built on an improved YOLOv5 network. The improved YOLOv5 network improves the feature extraction stage by introducing the Octave Conv module to reduce the computational load of the model; and it also improves the feature upsampling stage by introducing the CARAFE operator to enhance the target recognition and detection effect in complex situations.

[0012] Further improvements to the YOLOv5 network architecture include:

[0013] Input, Backbone network, Neck network, and prediction network;

[0014] The input end is used to receive and process infrared images to obtain the original feature image of the input model;

[0015] The Backbone network is used to process the original feature image through the Conv module, the C3 module based on the CSP module, the Octave Conv module, and the SPPF module to obtain several output feature maps;

[0016] The Neck network employs a composite structure of a path aggregation network and a feature pyramid network to further process several feature maps output by the Backbone network, resulting in several new feature maps.

[0017] The prediction network receives several feature maps output by the Neck network through several detection head modules, performs target detection, and outputs the corresponding detection results.

[0018] Furthermore, the structure of the Backbone network includes:

[0019] First, the original feature image is processed sequentially through the first Conv module, the second Conv module, the first C3 module, the third Conv module, and the second C3 module to obtain the first feature map;

[0020] Then, the first feature map is processed sequentially through the fourth Conv module and the third C3 module to obtain the second feature map;

[0021] Finally, the second feature map is processed sequentially through the fifth Conv module, the fourth C3 module, the first Octave Conv module, the SPPF module, and the second Octave Conv module to obtain the third feature map, which is used as the final output of the Backbone network.

[0022] Furthermore, the structure of the Neck network includes:

[0023] First, the third feature map is processed through the sixth Conv module to obtain the fourth feature map;

[0024] Then, after upsampling the fourth feature map using the first CARAFE operator, it is concatenated with the second feature map using the first Concat module to obtain the fifth feature map.

[0025] Then, the fifth feature map is processed by the fifth C3 module and the seventh Conv module to obtain the sixth feature map;

[0026] Then, after upsampling the sixth feature map using the second CARAFE operator, it is concatenated with the first feature map using the second Concat module to obtain the seventh feature map.

[0027] Then, the seventh feature map is processed by the sixth C3 module to obtain the eighth feature map;

[0028] Then, after the eighth feature map is processed by the eighth Conv module, it is concatenated with the sixth feature map by the third Concat module to obtain the ninth feature map.

[0029] Then, the ninth feature map is processed by the seventh C3 module to obtain the tenth feature map;

[0030] Then, after the tenth feature map is processed by the ninth Conv module, it is concatenated with the fourth feature map by the fourth Concat module to obtain the eleventh feature map.

[0031] Finally, the eleventh feature map is processed by the eighth C3 module to obtain the twelfth feature map;

[0032] The eighth, tenth, and twelfth feature maps are used as the final outputs of the Neck network.

[0033] Furthermore, the input end receives and processes the infrared image to obtain the original feature image of the input model, including:

[0034] Several different infrared images are stitched together into one image, and the stitched image is then subjected to data enhancement processing to obtain the enhanced image;

[0035] The optimal anchor box size is calculated based on the enhanced image for object detection in the prediction network;

[0036] An adaptive scaling operation is performed on the enhanced image to obtain the original feature image of the input model.

[0037] Furthermore, the prediction network employs the EIOU loss function, the expression of which is as follows:

[0038]

[0039] In the formula, This represents the value of the EIOU loss function. This indicates the intersection and union loss term. Represents the distance loss term. This represents the aspect ratio loss term. This represents the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. Represents Euclidean distance. and These represent the center coordinates of the predicted bounding box and the ground truth bounding box, respectively. and These represent the widths of the predicted bounding box and the ground truth bounding box, respectively. and These represent the heights of the predicted bounding box and the ground truth bounding box, respectively. , and These represent the diagonal length, width, and height of the smallest closure region of the predicted bounding box and the ground truth bounding box, respectively.

[0040] Furthermore, based on the temperature difference judgment criteria for different equipment defects, the specific steps for logically judging the target detection results and the corresponding temperature difference detection results to obtain the equipment defect type judgment result include:

[0041] For different types of power equipment, establish a temperature difference judgment rule library corresponding to the defects of different equipment; the temperature difference judgment rule library includes temperature value judgment rules and relative temperature difference judgment rules for different equipment under various defect states.

[0042] Based on the target detection results, determine the type of equipment detected, and select the corresponding temperature value judgment rule and relative temperature difference judgment rule from the temperature difference judgment rule library;

[0043] The temperature value at the location corresponding to the target detection result is compared with the selected temperature value judgment rule. If it meets the requirements of the temperature value judgment rule for a certain defect state, a preliminary defect type judgment result is obtained. If it does not meet the requirements, the subsequent steps are continued.

[0044] The temperature difference detection results are compared with the selected relative temperature difference judgment rules. If the results meet the requirements of the temperature difference judgment rules under a certain defect state, the upgraded defect type judgment result is obtained. If the results do not meet the requirements, the preliminary defect type judgment result is used as the final defect type judgment result.

[0045] Secondly, the present invention provides an infrared defect discrimination device based on an improved neural network, comprising:

[0046] The image acquisition module is used to acquire infrared images during power line inspection.

[0047] The image recognition module is used to identify infrared images using a pre-built infrared device image recognition model and an infrared temperature difference recognition model, respectively, to obtain target detection results and temperature difference detection results for each target; the temperature difference detection result is the temperature difference between the current temperature of the target and a preset reference temperature;

[0048] The defect identification module is used to make logical judgments on the target detection results and the corresponding temperature difference detection results based on the temperature difference judgment criteria of different equipment defects, and to obtain the defect type judgment result of the equipment.

[0049] Among them, the infrared device image recognition model is built on an improved YOLOv5 network. The improved YOLOv5 network improves the feature extraction stage by introducing the Octave Conv module to reduce the computational load of the model; and it also improves the feature upsampling stage by introducing the CARAFE operator to enhance the target recognition and detection effect in complex situations.

[0050] Thirdly, the present invention provides a computer device, the device including a processor and a memory:

[0051] The memory is used to store computer programs and send the instructions of the computer programs to the processor;

[0052] The processor executes an infrared defect detection method based on an improved neural network, as described in the first aspect, according to the instructions of the computer program.

[0053] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements an infrared defect discrimination method based on an improved neural network as described in the first aspect.

[0054] In summary, this invention provides an infrared defect discrimination method and related device based on an improved neural network. The method acquires infrared images during power line inspection; then, it uses a pre-built infrared equipment image recognition model and an infrared temperature difference recognition model to identify the infrared images, obtaining target detection results and temperature difference detection results for each target. The temperature difference detection result is the temperature difference between the target's current temperature and a pre-set reference temperature. Finally, based on the temperature difference judgment criteria for different equipment defects, a logical judgment is performed on the target detection results and the corresponding temperature difference detection results to obtain the equipment defect type determination result. The infrared equipment image recognition model is built based on an improved YOLOv5 network. In the feature extraction stage, the improved YOLOv5 network is improved by introducing the Octave Conv module to reduce the model's computational load. Furthermore, in the feature upsampling stage, the CARAFE operator is introduced to improve the target recognition and detection effect under complex conditions. This invention constructs an infrared device image recognition model based on an improved YOLOv5 network, combined with an infrared temperature difference recognition model, to automatically process infrared images from power inspections. This reduces manual interpretation, avoids missed or incorrect judgments due to visual fatigue, is not limited by personnel's professional skills, improves judgment efficiency, shortens processing time, and facilitates timely handling of infrared heating defects. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating an infrared defect discrimination method based on an improved neural network, provided in an embodiment of the present invention;

[0057] Figure 2 A technical roadmap for an infrared defect discrimination method based on an improved neural network, provided for embodiments of the present invention;

[0058] Figure 3 An improved YOLOv5 network structure diagram provided in this embodiment of the invention;

[0059] Figure 4 This is a structural diagram of the CSP module provided in an embodiment of the present invention;

[0060] Figure 5 This is a structural diagram of the SPPF module provided in an embodiment of the present invention;

[0061] Figure 6 The OctConv network structure provided in this embodiment of the invention;

[0062] Figure 7 The CARAFE module structure provided in this embodiment of the invention;

[0063] Figure 8 This is a structural diagram of Yolov5 provided in an embodiment of the present invention;

[0064] Figure 9 This is a basic residual block structure diagram provided for an embodiment of the present invention;

[0065] Figure 10 The ResNet structure diagram provided in the embodiments of the present invention;

[0066] Figure 11 A CBAM structure diagram provided for an embodiment of the present invention;

[0067] Figure 12 A flowchart of logical judgment provided for embodiments of the present invention;

[0068] Figure 13 A block diagram of an infrared defect discrimination device based on an improved neural network provided in an embodiment of the present invention;

[0069] Figure 14 This is a block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0070] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0071] Please see Figure 1 This invention provides an infrared defect discrimination method based on an improved neural network, comprising the following steps:

[0072] S11: Acquire infrared images during power line inspection.

[0073] It should be noted that infrared images are generated by converting the invisible infrared radiation emitted by an object into a visible image using an infrared detector. They are used in power line inspections to detect the heating status of equipment, because equipment defects often lead to localized temperature anomalies, which manifest as specific characteristics in infrared images.

[0074] This step utilizes infrared imaging equipment to photograph electrical equipment. During operation, the equipment generates heat changes due to defects (such as poor contact, overload, etc.), and the infrared imaging equipment converts these heat changes into infrared images.

[0075] S12: For infrared images, use the pre-built infrared device image recognition model and infrared temperature difference recognition model to identify them respectively, and obtain the target detection result and the temperature difference detection result of each target; the temperature difference detection result is the temperature difference between the current temperature of the target and the preset reference temperature.

[0076] Among them, the infrared device image recognition model is built on an improved YOLOv5 network. The improved YOLOv5 network improves the feature extraction stage by introducing the Octave Conv module to reduce the computational load of the model; and it also improves the feature upsampling stage by introducing the CARAFE operator to enhance the target recognition and detection effect in complex situations.

[0077] It should be noted that the infrared device image recognition model is a model built on an improved YOLOv5 network, used to identify devices and their defects in infrared images. YOLOv5 is a target detection algorithm capable of quickly detecting target objects in images, and the improved network further enhances performance. This model, based on the improved YOLOv5 network, extracts features from the infrared image through the Octave Conv module, obtaining feature maps at multiple scales. These features are then processed by the Neck network, and finally, the prediction network outputs the target detection results, determining the location and category of the device in the image.

[0078] Octave Conv is added before and after SPPF (Spatial Pyramid Pooling Module, introduced in a later section). It halves the spatial resolution of low-frequency components, performs convolution operations in two groups, and realizes the interaction of high and low frequency information through upsampling and downsampling. It removes spatial redundancy, saves model parameters and computation, and makes the convolution operation more lightweight.

[0079] The CARAFE (Content-Adaptive ReAssembly of Features) operator is used for feature upsampling operations. Traditional upsampling methods have small receptive fields and are prone to jagged edges, affecting target recognition and detection performance. CARAFE uses low-level content information to predict and reassemble the kernel and features of the surrounding region. Through the collaborative work of the kernel prediction module and the feature-aware reassembly module, the quality and efficiency of upsampling are improved, which helps to more accurately locate and identify targets.

[0080] Infrared temperature difference recognition models are used to calculate the temperature difference between the current temperature of a target and a preset reference temperature, helping to determine whether equipment has defects and the severity of those defects. Temperature difference is one of the important bases for judging equipment defects; different equipment defects will produce different temperature difference conditions.

[0081] S13: Based on the temperature difference judgment criteria for different equipment defects, perform logical judgment on the target detection results and the corresponding temperature difference detection results to obtain the equipment defect type judgment result.

[0082] It should be noted that different equipment defects correspond to specific temperature anomaly patterns (such as localized overheating or excessive temperature difference). Defect classification can be achieved through logical rule matching (such as temperature difference > a certain threshold indicating poor contact).

[0083] This embodiment provides an infrared defect discrimination method based on an improved neural network. First, infrared images of power line inspections are acquired. Then, an infrared equipment image recognition model constructed using an improved YOLOv5 network is used to detect and classify equipment in the images. Simultaneously, an infrared temperature difference recognition model is used to calculate the target temperature difference. The target detection results and temperature difference detection results are combined, and logical judgments are performed based on pre-established temperature difference judgment criteria to determine the defect type of the equipment. The improved YOLOv5 network, by introducing the Octave Conv module and the CARAFE operator, reduces computational load while improving target recognition and detection performance, thus enhancing the accuracy and efficiency of the entire method.

[0084] This method improves the YOLOv5 network by introducing the Octave Conv module, effectively reducing model computation and improving inference speed. It also introduces the CARAFE operator to enhance the quality and efficiency of feature upsampling, enabling the model to more accurately identify targets in complex situations. Furthermore, it combines infrared equipment image recognition and infrared temperature difference recognition models to analyze equipment from two dimensions: spatial location and temperature change, comprehensively judging equipment defect types and improving the accuracy and reliability of defect identification. Moreover, it uses temperature difference judgment as a basis for logically judging the detection results, making defect type determination more scientific and standardized, avoiding the subjectivity and uncertainty caused by manual judgment. In addition, this method reduces the workload of manually processing large amounts of image information, reduces missed or incorrect judgments caused by human visual fatigue, and improves the efficiency of power line inspection and the timeliness of defect handling.

[0085] Please see Figure 2 , Figure 2 This paper presents a technical approach for infrared defect detection based on an improved neural network, which mainly includes three parts: infrared device image recognition, infrared temperature difference recognition, and logical judgment. The above technical approach will be described below in conjunction with some other embodiments of the present invention.

[0086] First, the infrared equipment image recognition section identifies the type of primary power equipment and heat-generating components in the infrared image, such as the disconnector switch rotor and knife edge, the moving and stationary contacts and intermediate contacts of the circuit breaker, the internal connections of the current transformer, the bushing terminal head, and the capacitor fuse holder, etc. This includes the following steps:

[0087] Step 1: Acquire infrared images and category labels, and construct training and testing sample sets in a 5:2 ratio;

[0088] Step 2: Construct a YOLOv5 infrared image detection model by introducing Octconv into the backbone network and CARAFE module into the neck network to obtain an improved YOLOv5 network structure.

[0089] Step 3: Input the training sample set into the improved YOLOv5 object detection model for training, and obtain the model training weights;

[0090] Step 4: Input the test sample set into the trained YOLOv5 object detection model to obtain the corresponding object detection results.

[0091] In one embodiment of the present invention, an improved YOLO network is provided, the structure of which is as follows: Figure 3 As shown, the specific structure of the improved YOLOv5 network includes:

[0092] Input, Backbone network, Neck network, and prediction network;

[0093] The input end is used to receive and process infrared images to obtain the original feature image of the input model;

[0094] The Backbone network is used to process the original feature image through the Conv module, the C3 module based on the CSP module, the Octave Conv module, and the SPPF module to obtain several output feature maps;

[0095] The Neck network employs a composite structure of a path aggregation network and a feature pyramid network to further process several feature maps output by the Backbone network, resulting in several new feature maps.

[0096] The prediction network receives several feature maps output by the Neck network through several detection head modules, performs target detection, and outputs the corresponding detection results.

[0097] The Convolutional Module (Conv) is one of the most fundamental modules in deep learning. It performs convolutional operations on the input data through convolutional layers, using convolutional kernels to slide across the data and extract features. Convolutional operations can automatically extract local features from data such as images, including edges and textures, while reducing data dimensionality, computational cost, and model complexity. Activation functions (such as ReLU) are often connected after the convolutional layers to introduce non-linearity and enhance the model's expressive power.

[0098] The C3 module is based on CSPNet (Cross Stage Partial Network). It typically consists of multiple convolutional layers, batch normalization layers, and activation functions. Its key structural feature is that it divides the input feature map into two parts: one part undergoes direct convolutional operations, while the other part is processed through a series of CSP structures. Finally, the two results are fused. This design can reduce computational cost and parameter count while maintaining model performance, thus improving model efficiency.

[0099] The CSP module (Cross Stage Partial) shares weights across multiple stages of the network, reducing computation and the number of parameters while maintaining or improving model performance. By splitting feature maps by channel and processing them separately, it avoids redundant gradient calculations, improves model inference speed, alleviates the vanishing gradient problem, and enhances the network's feature extraction capabilities. The structure of the CSP module is as follows: Figure 4As shown, the core idea of ​​CSPNet is to share weights across multiple stages of the network to reduce computation and the number of parameters while maintaining or improving model performance. During network optimization, the problem of repeated gradient calculations arises, increasing the computational load. Therefore, to enhance the network's learning ability and reduce computation while maintaining accuracy, the CSP module is applied in feature extraction. CSP1_X is used in the backbone to reduce computation through weight sharing, while CSP2_X is applied to the Neck network, combining the advantages of CSP and Focus structures to further reduce computation while maintaining feature representation capabilities. CSP1_X first splits the feature map into two parts by channel. One part undergoes regular convolution, and the other part uses the concept of residual networks to construct residual components. Finally, these two parts are merged to obtain a new feature map. This design effectively avoids repeated gradient calculations, improving model inference speed. Simultaneously, the CSP structure, with its residual components, can enhance gradient values ​​during backpropagation, especially when the backbone network is deep, alleviating the gradient vanishing problem and enhancing the network's feature extraction capabilities. CSP2_X uses convolutional layers instead of residual components to divide the input feature map into two parts, calculate them separately, and then fuse them, which can retain more image information.

[0100] The Spatial Pyramid Pooling (SPPF) module is an improvement upon the traditional Spatial Pyramid Pooling (SPP) module. Traditional SPP concatenates feature maps after inputting them in parallel into max-pooling layers with different kernel sizes, while SPPF uses serial input of 5×5 kernel max-pooling layers. This achieves the same effect with lower computational cost and higher efficiency. The structure of SPPF is as follows: Figure 5 As shown in the diagram, in the traditional SPP module, feature maps are typically processed by parallel input max-pooling layers with kernel sizes of 5×5, 9×9, and 13×13, and then the four feature maps with different receptive sizes are concatenated. In contrast, the SPPF module uses serial input max-pooling layers with a kernel size of 5×5 before feature fusion. It is worth noting that since two 5×5 max-pooling layers have the same effect as a single 9×9 max-pooling layer, and three 5×5 max-pooling layers also produce the same result as a single 13×13 max-pooling layer, the SPPF structure achieves the same effect while reducing computational cost and increasing efficiency.

[0101] To address the issues of high computational cost and slow speed in network architecture, an improved network structure introduces Octave Conv (OctConv). OctConv is added before and after SPPF. Its core approach is to halve the spatial resolution of low-frequency components, then perform Conv in two groups. The two groups of frequencies interact through upsampling and downsampling, and finally synthesize the original feature map. By decomposing into low spatial resolution, spatial redundancy is removed, thereby saving model parameters and computational cost, making convolution operations more lightweight.

[0102] The OctConv network structure is as follows: Figure 6 As shown, suppose a feature map has c channels. in According to the preset coefficient a in Divided into high frequency (1-a) in ) c in With low frequency a in c in The two parts are divided into two sections; the width and height of the low-frequency portion are reduced to half of their original values. OctaveConvolution then performs the following four steps:

[0103] (1) Direct convolution of the high-frequency part: f(X) H That is, high-frequency to high-frequency convolution, with a certain number of output channels. ;

[0104] (2) The high-frequency part is first downsampled and then convolved. Here, the downsampling is pool(X). H , 2 ), then f (pool(X ), H , 2)) That is, high-frequency to low-frequency convolution, with an output channel number a out c out ;

[0105] (3) The low-frequency part is directly convolved and then upsampled: f(X) L Here, upsample(f(X) L This refers to convolution from low to high frequencies, with a certain number of output channels. ;

[0106] (4) Direct convolution of the low-frequency part: f(X) L That is, low-frequency to low-frequency convolution, with an output channel number a. out c out After these four parts are completed, the next step is to aggregate the information, that is, to perform a bitwise addition operation on the corresponding positions of the results of (1) and (3), and to perform a bitwise addition operation on the corresponding positions of the results of (2) and (4).

[0107] The upsampling operation in the YOLOv5 network has a small receptive field and is prone to jagged edges, which can affect target recognition and detection and fail to achieve optimal detection results. Therefore, an efficient and lightweight CARAFE operator is introduced to upsample the feature map.

[0108] CARAFE utilizes low-level content information to predict the reconstructed kernel and reassemble predefined nearby region features. This method can apply adaptive and optimized reconstructed kernels at different locations, significantly improving the quality and efficiency of upsampling. The CARAFE module structure is as follows: Figure 7 As shown, it consists of two parts: a kernel prediction module and a feature-aware reorganization module.

[0109] 1) Upsampling kernel prediction module

[0110] Step 1: Feature map channel compression; The input feature map has a shape and size of H×W×C. A 1×1 convolution is used to compress the number of channels from C to Cm to reduce the computational load of subsequent operations and improve upsampling efficiency.

[0111] Step 2: Encode the feature content and perform upsampling prediction; since different location information uses different upsampling kernels, let the upsampling kernel be k. up ×k up For the compressed feature map obtained in step 1, using k encoder ×k encoder The convolutional layer predicts the upsampling kernel, with Cm input channels and 100 output channels. Then, the feature image is expanded along the channel dimension to obtain a size of upsampling kernel (where (This refers to the upsampling factor).

[0112] Step 3: Normalization; The upsampled kernels obtained from the prediction in Step 2 are normalized using the Softmax operation to ensure that the weights of the convolution kernels sum to 1.

[0113] 2) Feature-aware reassembly module

[0114] This mainly involves mapping and dot product operations. First, each position in the output feature map is mapped back to the input feature map, forming a new feature map. Next, for the center of each position, extract k from the center of each position. up ×k up Regions and Recombined Kernel W l A dot product operation is performed to allow upsampling kernels with the same upsampling kernel to share information across different channels at the same location, ultimately yielding the size. upsampled feature map This process recombines the feature maps through dot product operations, allowing the upsampling kernel to be shared across different channels and resulting in more adaptive upsampling feature maps, thus providing an effective means to further improve model performance.

[0115] The CARAFE module, through the collaborative work of these two modules, can use adaptive and optimized recombination kernels at different locations to improve the quality and efficiency of upsampling. This design helps overcome the problems encountered in traditional upsampling methods, especially in target detection tasks, where it significantly improves the accurate location and identification of defective targets.

[0116] The recognition model has feature maps at three scales: 80x80, 40x40, and 20x20 grids. Different grids correspond to different sizes of detection targets. By using the NMS (Non-Maximum Suppression) algorithm, the model can effectively eliminate redundant predicted boxes that do not meet the threshold conditions, thereby enhancing the target recognition capability and completing the entire target detection process.

[0117] This embodiment proposes an improved YOLOv5 network for target detection in infrared images. Its overall structure consists of four parts: an input network, a backbone network, a neck network, and a prediction network. The input layer receives infrared images and transforms them into raw feature images suitable for model processing through data augmentation, adaptive anchor box calculation, and adaptive image scaling, laying the foundation for subsequent processing. The Backbone network uses Conv modules, a C3 module based on the CSP module, an Octave Conv module, and an SPPF module to perform multi-level feature extraction on the raw feature images, obtaining several output feature maps containing rich semantic and spatial information. These feature maps are the core data for subsequent detection. The Neck network uses a composite structure of Path Convergence Network (PAN) and Feature Pyramid Network (FPN) to further process the feature maps output by the Backbone network. Through bidirectional feature fusion, information flow between feature maps of different scales is achieved, enhancing feature representation capabilities and obtaining several new feature maps. The prediction network receives the feature maps output by the Neck network through several detection head modules, analyzes the feature maps using internal classification and regression layers, classifies and locates the target, and finally outputs the target detection result. This improved YOLOv5 network, through the collaborative work of its various components, can more efficiently and accurately achieve target detection in infrared images.

[0118] In a further embodiment of the present invention, an improved input design for the YOLOv5 network is proposed. The input terminal receives and processes infrared images to obtain the original feature image of the input model, including:

[0119] S21: Stitch together several different infrared images into one image, and perform data enhancement processing on the stitched image to obtain the enhanced image.

[0120] Data augmentation can be performed using Mosaic data augmentation. Mosaic data augmentation stitches together four different images by randomly cropping, scaling, and arbitrarily arranging them into a single image. If there are blank areas in the stitched image, these areas will be filled with a specific pixel value (114 in this example). The stitched image is then further augmented as input to the detector.

[0121] S22: Calculate the optimal anchor box size based on the enhanced image for object detection in the prediction network.

[0122] The anchor box size in this step can be calculated using adaptive anchor box calculation, which means that the optimal anchor box size is automatically calculated during model training, eliminating the need to calculate the anchor box size separately for different datasets. This calculation allows the model to better adapt to the characteristics of different datasets, improving the model's generalization ability.

[0123] S23: Perform adaptive scaling on the enhanced image to obtain the original feature image of the input model.

[0124] Please refer to it again. Figure 3 In a further embodiment of the present invention, an improved backbone network design for a YOLOv5 network is proposed, the structure of which includes:

[0125] First, the original feature image is processed sequentially through the first Conv module, the second Conv module, the first C3 module, the third Conv module, and the second C3 module to obtain the first feature map;

[0126] Then, the first feature map is processed sequentially through the fourth Conv module and the third C3 module to obtain the second feature map;

[0127] Finally, the second feature map is processed sequentially through the fifth Conv module, the fourth C3 module, the first Octave Conv module, the SPPF module, and the second Octave Conv module to obtain the third feature map, which is used as the final output of the Backbone network.

[0128] Please refer to it again. Figure 3 In a further embodiment of the present invention, an improved neck network design for the YOLOv5 network is proposed, the structure of which includes:

[0129] First, the third feature map is processed through the sixth Conv module to obtain the fourth feature map;

[0130] Then, after upsampling the fourth feature map using the first CARAFE operator, it is concatenated with the second feature map using the first Concat module to obtain the fifth feature map.

[0131] Then, the fifth feature map is processed by the fifth C3 module and the seventh Conv module to obtain the sixth feature map;

[0132] Then, after upsampling the sixth feature map using the second CARAFE operator, it is concatenated with the first feature map using the second Concat module to obtain the seventh feature map.

[0133] Then, the seventh feature map is processed by the sixth C3 module to obtain the eighth feature map;

[0134] Then, after the eighth feature map is processed by the eighth Conv module, it is concatenated with the sixth feature map by the third Concat module to obtain the ninth feature map.

[0135] Then, the ninth feature map is processed by the seventh C3 module to obtain the tenth feature map;

[0136] Then, after the tenth feature map is processed by the ninth Conv module, it is concatenated with the fourth feature map by the fourth Concat module to obtain the eleventh feature map.

[0137] Finally, the eleventh feature map is processed by the eighth C3 module to obtain the twelfth feature map;

[0138] The eighth, tenth, and twelfth feature maps are used as the final outputs of the Neck network.

[0139] In a further embodiment of the present invention, in order to effectively evaluate the positional loss of the target box and the predicted box, the EIOU loss function is adopted to avoid the problem of slow convergence speed caused by the non-overlapping bounding boxes. EIOU not only considers the effects of the overlapping area and center point distance already covered by the CIOU loss function, but also the effect of the side length, as expressed below:

[0140]

[0141] In the formula, This represents the value of the EIOU loss function. This indicates the intersection and union loss term. Represents the distance loss term. This represents the aspect ratio loss term. This represents the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. Represents Euclidean distance. and These represent the center coordinates of the predicted bounding box and the ground truth bounding box, respectively. and These represent the widths of the predicted bounding box and the ground truth bounding box, respectively. and These represent the heights of the predicted bounding box and the ground truth bounding box, respectively. , and These represent the diagonal length, width, and height of the smallest closure region of the predicted bounding box and the ground truth bounding box, respectively.

[0142] Secondly, the infrared temperature difference recognition section aims to identify the temperature in the image and obtain the temperature difference data of the heating points, which serves as the second input data for the subsequent logic decision unit. This includes the following steps:

[0143] Step 1: Acquire the first infrared image during the power line inspection process;

[0144] Step 2: Perform image enhancement on the acquired infrared image to obtain a second infrared image;

[0145] Step 3: Introduce the YOLO object detection algorithm to locate the bounding boxes in the image, and use the ResNet convolutional neural network to identify the temperature within the bounding boxes, obtaining the relative temperature difference and hotspot temperature within the bounding boxes;

[0146] Step 4: Input the obtained relative temperature difference value and target detection result into the logic judge.

[0147] The infrared temperature difference recognition model is built based on the YOLOv5 target detection algorithm. It uses the YOLOv5 algorithm to locate the fractional bounding box. The algorithm structure is as follows: Figure 8 As shown, it consists of four parts: input, backbone, neck, and head.

[0148] Score recognition is achieved based on a ResNet (Residual Network) 18 convolutional neural network, which is mainly composed of basic residual blocks. Figure 9 (a) represents the basic residual block I, which contains a straight path and a skip path. The straight path consists of two 3×3 convolutional layers and two batch normalization layers, activated by the ReLU function in between. The skip path directly adds the input feature map to the output of the straight path. Finally, the sum is activated by the ReLU function and then output. Figure 9(b) is the basic residual block II, which is similar in structure to the ordinary residual block. The difference is that the skip path adds a 1×1 convolutional layer and a BN (Batch Normalization) layer to downsample the input feature map.

[0149] The ResNet 18 architecture is as follows: Figure 10 As shown, the input image is first subjected to 7×7 convolution and max pooling operations, then features are extracted through 8 basic residual blocks, and finally the predicted probability is output through average pooling and fully connected layers.

[0150] The Convolutional Block Attention Module (CBAM) is integrated into the ResNet network, added to the second and penultimate layers to improve the network's feature extraction capabilities in both channels and spatial dimensions. The CBAM is a lightweight attention module with the following structure: Figure 11 As shown, it includes a channel attention module (CAM) and a spatial attention module (SAM).

[0151] Finally, for the logical judgment part, the temperature difference judgment criteria for three types of defects in different primary equipment types and locations from the infrared testing guidelines are imported (where defect nature types are divided into three categories: general defects, major defects, and critical defects) to construct a logical judgment unit for defect type identification. The logical judgment process is as follows: Figure 12 As shown, it includes:

[0152] S31: Establish a temperature difference judgment rule library for different types of power equipment, corresponding to different equipment defects; the temperature difference judgment rule library includes temperature value judgment rules and relative temperature difference judgment rules for different equipment under various defect states;

[0153] S32: Determine the type of equipment detected based on the target detection results, and select the corresponding temperature value judgment rule and relative temperature difference judgment rule from the temperature difference judgment rule library;

[0154] S33: Compare the temperature value at the location corresponding to the target detection result with the selected temperature value judgment rule (that is, compare the hot spot temperature with the temperature value set under this equipment type). If it meets the temperature value judgment rule requirements under a certain defect state (for example, the hot spot temperature is greater than the temperature value set under this equipment type), a preliminary defect type judgment result is obtained. If it does not meet the requirements, a defect upgrade judgment is performed.

[0155] S34: Compare the temperature difference detection result with the selected relative temperature difference judgment rule (that is, compare the relative temperature difference with the relative temperature difference set under this equipment type). If it meets the requirements of the temperature difference judgment rule under a certain defect state (e.g., the relative temperature difference is greater than the relative temperature difference set under this equipment type), the upgraded defect type judgment result is obtained. If it does not meet the requirements, the preliminary defect type judgment result is used as the final defect type judgment result.

[0156] This invention proposes an infrared defect identification method. The constructed identification model boasts high accuracy and low computational cost, making it adaptable to the complex environment of power equipment identification within substations. It improves identification accuracy while accelerating the identification speed. Based on the judgment criteria of infrared testing guidelines, a logic determiner is constructed. Image recognition and temperature extraction technologies are used as input data for the logic determiner to obtain the type of infrared defect. This method can efficiently and accurately identify infrared defect types, reducing manual analysis costs and improving analysis accuracy.

[0157] Based on the same inventive concept, this application also provides an infrared defect discrimination device based on an improved neural network for implementing the infrared defect discrimination method based on the improved neural network described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations in the embodiments of the infrared defect discrimination device based on the improved neural network provided below can be found in the limitations of the infrared defect discrimination method based on the improved neural network described above, and will not be repeated here.

[0158] Please see Figure 13 This invention also provides an infrared defect discrimination device based on an improved neural network, comprising:

[0159] The image acquisition module is used to acquire infrared images during power line inspection.

[0160] The image recognition module is used to identify infrared images using a pre-built infrared device image recognition model and an infrared temperature difference recognition model, respectively, to obtain target detection results and temperature difference detection results for each target; the temperature difference detection result is the temperature difference between the current temperature of the target and a preset reference temperature;

[0161] The defect identification module is used to make logical judgments on the target detection results and the corresponding temperature difference detection results based on the temperature difference judgment criteria of different equipment defects, and to obtain the defect type judgment result of the equipment.

[0162] Among them, the infrared device image recognition model is built on an improved YOLOv5 network. The improved YOLOv5 network improves the feature extraction stage by introducing the Octave Conv module to reduce the computational load of the model; and it also improves the feature upsampling stage by introducing the CARAFE operator to enhance the target recognition and detection effect in complex situations.

[0163] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0164] Reference Figure 14 The present invention also provides a computer device, including: a memory and a processor, and a computer program stored in the memory, wherein when the computer program is executed on the processor, it implements the infrared defect discrimination method based on the improved neural network as described in any of the above methods.

[0165] The computer device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 14 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. They may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, they may also include input / output devices, network access devices, etc.

[0166] The processor referred to can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0167] In some embodiments, the memory may be an internal storage unit of the computer device, such as a hard drive or RAM. In other embodiments, the memory may be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory may include both internal and external storage units of the computer device. The memory is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or will be output.

[0168] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the infrared defect discrimination method based on an improved neural network as described in any of the above methods.

[0169] In this embodiment, if the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0170] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0171] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0172] In the embodiments disclosed in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0173] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for infrared defect discrimination based on an improved neural network, characterized in that, Includes the following steps: Acquire infrared images during power line inspection; The infrared image is identified using a pre-built infrared device image recognition model and an infrared temperature difference recognition model to obtain target detection results and temperature difference detection results for each target; the temperature difference detection results are the temperature difference between the current temperature of the target and a preset reference temperature. Based on the temperature difference judgment criteria for different equipment defects, logical judgment is performed on the target detection result and the corresponding temperature difference detection result to obtain the equipment defect type judgment result; The infrared device image recognition model is built on an improved YOLOv5 network. In the feature extraction stage, the improved YOLOv5 network is improved by introducing the Octave Conv module to reduce the computational load of the model. In the feature upsampling stage, the CARAFE operator is introduced to improve the target recognition and detection effect in complex situations.

2. The infrared defect discrimination method based on an improved neural network according to claim 1, characterized in that, The structure of the improved YOLOv5 network includes: Input, Backbone network, Neck network, and prediction network; The input terminal is used to receive and process the infrared image to obtain the original feature image of the input model; The Backbone network is used to process the original feature image through the Conv module, the C3 module based on the CSP module, the Octave Conv module, and the SPPF module to obtain several output feature maps; The Neck network employs a composite structure of a path aggregation network and a feature pyramid network to further process several feature maps output by the Backbone network, thereby obtaining several new feature maps. The prediction network is used to receive several feature maps output by the Neck network through several detection head modules, and output the corresponding detection results after performing target detection.

3. The infrared defect discrimination method based on an improved neural network according to claim 2, characterized in that, The structure of the Backbone network includes: First, the original feature image is processed sequentially through the first Conv module, the second Conv module, the first C3 module, the third Conv module, and the second C3 module to obtain the first feature map; Then, the first feature map is processed sequentially through the fourth Conv module and the third C3 module to obtain the second feature map; Finally, the second feature map is processed sequentially through the fifth Conv module, the fourth C3 module, the first Octave Conv module, the SPPF module, and the second Octave Conv module to obtain the third feature map, which serves as the final output of the Backbone network.

4. The infrared defect discrimination method based on an improved neural network according to claim 3, characterized in that, The structure of the Neck network includes: First, the third feature map is processed by the sixth Conv module to obtain the fourth feature map; Then, after upsampling the fourth feature map using the first CARAFE operator, it is concatenated with the second feature map using the first Concat module to obtain the fifth feature map. Then, the fifth feature map is processed by the fifth C3 module and the seventh Conv module to obtain the sixth feature map; Then, after upsampling the sixth feature map using the second CARAFE operator, it is concatenated with the first feature map by the second Concat module to obtain the seventh feature map. Then, the seventh feature map is processed by the sixth C3 module to obtain the eighth feature map; Then, after the eighth feature map is processed by the eighth Conv module, it is concatenated with the sixth feature map by the third Concat module to obtain the ninth feature map. Then, the ninth feature map is processed by the seventh C3 module to obtain the tenth feature map; Then, after the tenth feature map is processed by the ninth Conv module, it is concatenated with the fourth feature map by the fourth Concat module to obtain the eleventh feature map. Finally, the eleventh feature map is processed by the eighth C3 module to obtain the twelfth feature map; The eighth feature map, the tenth feature map, and the twelfth feature map serve as the final output of the Neck network.

5. The infrared defect discrimination method based on an improved neural network according to claim 2, characterized in that, The input terminal receives and processes the infrared image to obtain the original feature image of the input model, including: Several different infrared images are stitched together into one image, and the stitched image is then subjected to data enhancement processing to obtain an enhanced image; The optimal anchor box size is calculated based on the enhanced image for target detection in the prediction network; An adaptive scaling operation is performed on the enhanced image to obtain the original feature image of the input model.

6. The infrared defect discrimination method based on an improved neural network according to claim 2, characterized in that, The prediction network uses the EIOU loss function, the expression of which is as follows: In the formula, This represents the value of the EIOU loss function. This indicates the intersection and union loss term. Represents the distance loss term. This represents the aspect ratio loss term. This represents the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. Represents Euclidean distance. and These represent the center coordinates of the predicted bounding box and the ground truth bounding box, respectively. and These represent the widths of the predicted bounding box and the ground truth bounding box, respectively. and These represent the heights of the predicted bounding box and the ground truth bounding box, respectively. , and These represent the diagonal length, width, and height of the smallest closure region of the predicted bounding box and the ground truth bounding box, respectively.

7. The infrared defect discrimination method based on an improved neural network according to claim 1, characterized in that, Based on the temperature difference judgment criteria for different equipment defects, the specific steps for logically judging the target detection result and the corresponding temperature difference detection result to obtain the equipment defect type judgment result include: For different types of power equipment, a temperature difference judgment rule library is established for different equipment defects; the temperature difference judgment rule library includes temperature value judgment rules and relative temperature difference judgment rules for different equipment under various defect states. Based on the target detection results, the detected device type is determined, and the corresponding temperature value judgment rule and relative temperature difference judgment rule are selected from the temperature difference judgment rule library; The temperature value at the location corresponding to the target detection result is compared with the selected temperature value judgment rule. If it meets the requirements of the temperature value judgment rule under a certain defect state, a preliminary defect type judgment result is obtained. If it does not meet the requirements, the subsequent steps are continued. The temperature difference detection result is compared with the selected relative temperature difference judgment rule. If it meets the requirements of the temperature difference judgment rule under a certain defect state, the upgraded defect type judgment result is obtained. If it does not meet the requirements, the preliminary defect type judgment result is taken as the final defect type judgment result.

8. An infrared defect detection device based on an improved neural network, characterized in that, include: The image acquisition module is used to acquire infrared images during power line inspection. The image recognition module is used to recognize the infrared image using a pre-built infrared device image recognition model and an infrared temperature difference recognition model, respectively, to obtain target detection results and temperature difference detection results for each target; the temperature difference detection results are the temperature difference between the current temperature of the target and a preset reference temperature. The defect identification module is used to make logical judgments on the target detection results and the corresponding temperature difference detection results based on the temperature difference judgment criteria of different equipment defects, so as to obtain the defect type judgment result of the equipment. The infrared device image recognition model is built on an improved YOLOv5 network. In the feature extraction stage, the improved YOLOv5 network is improved by introducing the Octave Conv module to reduce the computational load of the model. In the feature upsampling stage, the CARAFE operator is introduced to improve the target recognition and detection effect in complex situations.

9. A computer device, characterized in that, The device includes a processor and a memory: The memory is used to store computer programs and send the instructions of the computer programs to the processor; The processor executes an infrared defect discrimination method based on an improved neural network as described in any one of claims 1-7 according to the instructions of the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements an infrared defect discrimination method based on an improved neural network as described in any one of claims 1-7.

Images