An edge-computing-based power transmission and transformation equipment infrared image target recognition method, electronic device and storage medium

Abstract

The application provides an infrared image target recognition method for power transmission and transformation equipment based on edge computing, an electronic device and a storage medium. The infrared image of the power transmission and transformation equipment to be recognized is acquired, and a first image of a fixed size is obtained through preprocessing. The first image is recognized according to a predetermined target recognition network model. The predetermined target recognition network model is based on a YOLOv7 network framework and is deployed in an edge computing node. The model includes a backbone network Backbone and a detection head Head. The recognition result of the first image is obtained. The recognition result is a target prediction box generated by the predetermined target recognition network model. Compared with the prior art, the improved CA module adds a residual structure, which improves the information preservation capability and increases the feature representation of the corresponding object, thereby providing more effective features for the prediction of the Head part. The CoT module is added in the three Detect detectors to learn the context information of the input features and improve the final classification accuracy.

Description

Technical Field

[0001] This invention relates to the field of target recognition technology, and more specifically, to a method, electronic device, and storage medium for infrared image target recognition of power transmission and transformation equipment based on edge computing. Background Technology

[0002] For the identification of power transmission and transformation equipment, traditional algorithms struggle to design a universal model to extract features from different devices. Furthermore, differences in infrared image capture angles, lighting conditions, and complex background interference significantly limit the accuracy of identifying complex objects. In addition, traditional algorithms consume a considerable amount of time, hindering the rapid and accurate analysis of the equipment.

[0003] Currently, deep learning-based methods mainly use models such as YOLO series or R-CNN (Region-based Convolution Neural Networks) to identify and detect power transmission and transformation equipment. However, although R-CNN has high accuracy, it divides the network into two stages, which is cumbersome. At the same time, when extracting images, there is some redundancy due to the large number of generated candidate regions, which increases the completion time and also increases the difficulty of extracting candidate boxes in complex scenes. Therefore, it cannot well meet the needs of some real-time scenarios. Summary of the Invention

[0004] In view of this, the purpose of this embodiment of the invention is to provide a model for generating a recognition model by adding an improved CA module and a CoT module to the YOLOv7 network, thereby improving the network's representation ability and the classification accuracy of power transmission and transformation equipment; the model trained using this network can effectively complete the infrared image target recognition task of power transmission and transformation equipment, thereby enabling real-time monitoring of the status of power transmission and transformation equipment, which is beneficial to improving the operational reliability and resource utilization of power transmission and transformation equipment.

[0005] A first aspect of the present invention provides a method for infrared image target recognition of power transmission and transformation equipment based on edge computing, the method comprising:

[0006] Acquire infrared images of the power transmission and transformation equipment to be identified, and perform preprocessing to scale them into a first image of a fixed size;

[0007] The first image is identified based on a target recognition network model deployed in edge computing nodes. The predetermined target recognition network model is based on the YOLOv7 network framework and includes a backbone network and a head. The backbone network includes two CA modules, which are located after two convolutional modules (Conv). Each of the three Detect modules in the head incorporates a CoT module, which fuses feature maps representing static and dynamic contexts as output.

[0008] Obtain the recognition result of the first image; the recognition result is a target prediction box generated by a predetermined target recognition network model.

[0009] Preferably, the CA module is used to perform one-dimensional horizontal global pooling and one-dimensional vertical global pooling on the input feature information, and then map the input to the range (0, 1) based on channel addition, two-dimensional convolution, normalization and activation operations so that the Re-weight module can assign weights to the residual connection information.

[0010] Preferably, the CoT module is used to map the static context representation of the input data X to K. 1 And based on the attention matrix A of the context, a new feature map K is generated. 2 ; Set the static context K 1 and dynamic context K 2 The output of the CoT module is generated through fusion using an attention mechanism.

[0011] Preferably, the predetermined target recognition network model further includes YOLOv7 built-in modules; the YOLOv7 built-in modules include ELAN, ELAN-H, MPConv, SPPCSPC, and RepConv; ELAN consists of multiple convolutional layers, the input and output feature sizes remain unchanged, the number of channels changes after passing through two convolutions, and the output after the last convolution is the required number of channels; ELAN-H also consists of multiple convolutional layers, and its operation takes into account the results of the previous two convolutions when concatenating at the end of ELAN;

[0012] The MPConv module has the same number of input and output channels, but the output size is half the input size. The upper part is halved by MaxPool and then halved by convolution. The lower part is halved by convolution and then halved by a convolution with a stride of 2. The upper and lower parts are then merged by cat to obtain the output.

[0013] The SPPCSPC network structure mainly consists of convolutional layers and a MaxPool. The output layer channel of the entire SPPCSPC layer is out_c. During training, a hidden layer channel, hidden_c = int(2*e*out_c), is calculated to expand the information content. Generally, e = 0.5 is taken, so hidden_c = out_c.

[0014] Preferably, RepConv uses different structures during training and inference, and employs model reparameterization techniques. During training, RepConv consists of a 3x3 convolutional branch and a 1x1 convolutional branch. If the number of input and output channels and their sizes are the same, a branch with only a BN layer is added, and the three branches are summed for output. During inference, to improve efficiency, the parameters of the branches are reparameterized onto the main branch, and the 3x3 main branch convolution is used for output.

[0015] Preferably, the method further includes a training process for a predetermined target recognition network model; the training process includes: generating a training set and a test set using infrared images of several power transmission and transformation equipment; setting performance indicators such as accuracy, recall, and IOU; and training and optimizing the model based on preset recognition accuracy and the ratio of IOU between the predicted box and the actual box, so that the overlap between the predicted box and the actual equipment's labeled box is high.

[0016] Furthermore, a second aspect of the present invention provides an electronic device comprising: one or more processors, and a memory for storing one or more computer programs; characterized in that the computer programs are configured to be executed by the one or more processors, the programs comprising steps for performing the edge computing-based infrared image target recognition method for power transmission and transformation equipment as described in the first aspect.

[0017] Furthermore, a third aspect of the present invention provides a storage medium storing a computer program; the program is loaded and executed by a processor to implement the steps of the edge computing-based infrared image target recognition method for power transmission and transformation equipment as described in the first aspect.

[0018] In this invention, an infrared image of the power transmission and transformation equipment to be identified is acquired and preprocessed to be scaled into a first image of a fixed size. The first image is then identified according to a predetermined target recognition network model. The predetermined target recognition network model is based on the YOLOv7 network framework and includes a backbone network and a head. The backbone network includes two CA modules, which are located after two convolutional modules (Conv). A CoT module is added to each of the three Detect modules in the head, and the CoT module fuses the feature maps representing static and dynamic contexts as output. The recognition result of the first image is obtained, and the recognition result is a target prediction box generated by the predetermined target recognition network model. Compared to existing technologies, this paper proposes an improved YOLOv7 infrared image target recognition algorithm, CACoT-YOLOv7. Based on YOLOv7, this algorithm adds an improved CA module to the YOLOv7 backbone. Compared to the original CA module, it incorporates a residual structure, enhancing information retention while increasing feature representation of the corresponding objects, thus providing more effective features for head prediction. Furthermore, a CoT module is added to the three detectors to learn the contextual information of the input features, improving the final classification accuracy. This enables high-precision target recognition of power transmission and transformation equipment such as insulators, conductors, fittings, and transformer bushings, thereby improving resource utilization efficiency and the automation and intelligence level of substations. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the overall structure of the target recognition network model disclosed in an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the structure of the CA module disclosed in an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the CoT module disclosed in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the performance indicators disclosed in the embodiments of the present invention;

[0024] Figure 5This is a schematic diagram of the predicted image of the target recognition network model disclosed in an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram of the edge computing architecture disclosed in an embodiment of the present invention. Detailed Implementation

[0026] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0027] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0028] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0029] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0030] It should be noted that "multiple" as mentioned in this article refers to two or more.

[0031] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0032] Definitions of terms used in this embodiment:

[0033] Object detection model (YouOnly Look Oncev7, YOLOv7)

[0034] Power transmission and transformation equipment

[0035] Infrared image (IR)

[0036] Object Detection

[0037] Deep learning (DL)

[0038] Rapid national development has led to a continuous increase in electricity demand, and the widespread deployment of power equipment across the country has increased the probability of equipment failures. Due to the increasing frequency of accidents caused by the power system, my country is placing greater emphasis on the safety and automation of power equipment. Transmission and transformation equipment is a crucial component of the power system, and its main failures manifest as mechanical damage, temperature increases, and localized changes in the electric field. Traditional equipment monitoring requires experienced personnel, is costly, and prone to significant errors, increasing the difficulty of intelligent power inspection. Non-contact equipment monitoring can effectively address this problem. Non-contact equipment monitoring primarily analyzes equipment conditions through infrared image analysis; its wide temperature measurement range and high accuracy have led to its widespread application in the monitoring of transmission and transformation equipment.

[0039] With the continuous development of infrared diagnostic technology, its application in temperature-related aspects of power transmission and transformation equipment has become increasingly widespread. Infrared images of the equipment can be used to intuitively identify overheating issues. Simultaneously, with the maturation of target detection technology, combining processed infrared images of power transmission and transformation equipment with target detection algorithms enables rapid fault diagnosis. Deep learning-based target detection methods are currently applied in various image recognition fields. Applying this method to power transmission and transformation equipment detection can significantly improve the accuracy and speed of identification, and facilitate subsequent analysis of the equipment, thereby ensuring the safe and stable operation of power transmission and transformation equipment and further improving the safety factor of substations. Therefore, using deep learning-based target detection methods to identify power transmission and transformation equipment in substations has significant advantages. Detection models trained using infrared image data can automatically identify a large number of substation infrared images, thereby reducing costs and effectively ensuring the safety, social, and economic benefits of substations.

[0040] Current power transmission and transformation equipment detection algorithms are mainly divided into two types: traditional methods and deep learning-based methods. Traditional object detection algorithms mainly include Cascade+HOG / DPM+Haar, and many other detection algorithms are improvements on these algorithms. Generally speaking, they extract edge information of various devices through algorithms, i.e., feature extraction. After obtaining these features, they are combined with algorithms such as Support Vector Machine (SVM) and Adaptive Boosting to classify different devices. Deep learning-based methods mainly use models based on the YOLO series or R-CNN algorithms. R-CNN generates many small regions on the image through segmentation algorithms, merges these small regions based on their color and structural features to obtain candidate boxes, and then performs object detection and classification through convolutional neural networks and fully connected layers after unifying the image size. YOLO series algorithms generally convert the input image into a 7*7 grid, then predict these grids using two bounding boxes, and finally filter the prediction results of the bounding boxes using maximum suppression to obtain the final predicted boxes. Regardless of the specific deep learning-based method, they generally involve training on infrared images labeled with various device information or training a pre-trained model on other datasets before transferring the training to a new model. This model enables more accurate and rapid identification and positioning of power transmission and transformation equipment.

[0041] For the identification of power transmission and transformation equipment, traditional algorithms struggle to design a universal model to extract features from different devices. Furthermore, differences in infrared image capture angles, lighting conditions, and complex background interference significantly limit the accuracy of identifying complex objects. In addition, traditional algorithms consume considerable time, hindering rapid and accurate subsequent equipment analysis. Currently, deep learning-based methods primarily utilize models like the YOLO series or R-CNN for power transmission and transformation equipment identification and detection. While R-CNN offers high accuracy, its two-stage, complex process and the large number of generated candidate regions during image extraction introduce redundancy, increasing processing time and complicating candidate box extraction in complex scenes. Therefore, it cannot adequately meet the real-time requirements of some scenarios. For the YOLO series, it eliminates the need for a two-stage network, enabling target detection and classification directly through a single network. YOLOv2, compared to YOLOv1, added a prior box regression mechanism and used the new network architecture Darknet-19. YOLOv3, compared to YOLOv2, added a residual network and utilized multi-scale feature fusion for target detection. YOLOv4 and YOLOv3 are essentially similar, primarily introducing the CSP block (CSP module) to improve accuracy. YOLOv5, based on the PyTorch framework, is faster and has a more user-friendly configuration file. YOLOv7 uses an efficient aggregation network combined with reparameterization to accelerate the network while maintaining model performance, surpassing previously developed detectors in both speed and accuracy within the 5FPS to 160FPS range.

[0042] In addition, some works combine the YOLO series with attention mechanisms to improve the network's recognition capabilities. These attention mechanisms include the Squeeze-and-Excitation block (SE), the Bottleneck Attention Module (BAM), and the Convolutional Block Attention Module (CBAM). However, the SE mechanism only considers the encoding of inter-channel information, ignoring the importance of positional information, which is crucial for capturing target structure in visual tasks. The BAM and CBAM mechanisms attempt to utilize positional information to reduce the channel dimension of the input tensor and then use convolution to compute spatial attention. However, convolution can only capture local relationships and cannot model long-term dependencies. The Coordinate Attention (CA) mechanism embeds positional information into channel attention, enabling mobile networks to focus over a larger area while avoiding significant computational overhead and improving classification accuracy. Contextual Transformer (CoT) is a self-attention mechanism that, compared to the self-attention mechanisms used in Transformer, fully utilizes the contextual information between input keys to guide the learning of dynamic attention matrices, thereby enhancing the ability of visual representation.

[0043] However, with the continuous input of infrared images from power transmission and transformation equipment in practical applications, the computing demands on servers will continue to increase. To address the drawbacks of traditional data processing methods, such as high latency and insufficient real-time data analysis capabilities, edge computing technology has emerged. Edge computing technology provides edge intelligent services locally at the network edge, close to the source of the data or the object itself, through a distributed open platform that integrates core capabilities of networking, computing, storage, and applications. Simply put, edge computing allows data collected from terminals to be analyzed directly on the local device or network where the data was generated, without needing to transmit the data to a cloud data processing center.

[0044] To achieve high-precision target recognition of power transmission and transformation equipment such as insulators, conductors, fittings, and transformer bushings, this embodiment provides a target recognition algorithm for infrared images of power transmission and transformation equipment based on an improved YOLOv7, called CACoT-YOLOv7. It is based on the YOLOv7 network framework, with the addition of an improved CA module and a CoT module. The improved CA module enhances the representation capability of the backbone, while the CoT module utilizes features provided by the backbone to complete self-attention learning during the head prediction process. Combining these two attention mechanisms with YOLOv7 improves the network's classification and recognition capabilities, enabling accurate localization and recognition of various types of power transmission and transformation equipment in infrared images. Furthermore, deploying the trained model to edge computing nodes effectively reduces deployment costs and improves information security.

[0045] like Figure 1 The diagram shown is a schematic representation of the overall structure of the target recognition network model in this embodiment.

[0046] Specifically, from the overall network structure, the input is mainly divided into two parts: the Backbone and the Head. The Backbone is used for feature extraction, and the Head is used for prediction. The Conv module in the Backbone consists of two sets of Conv+BN+SiLU. These two sets are identical except for the stride of the Conv layer. Conv+BN+SiLU represents a convolutional layer + normalization + SiLU activation layer.

[0047] In terms of process, the image is first resized to a fixed size and input into the Backbone network. Then, based on the feature maps of three different sizes in the Backbone network, the Head section continues to output the final result after passing through the RepConv module to predict the bounding boxes. Size refers to the width and height of the feature map, and resize refers to adjusting the size of the input image.

[0048] The Bacbone section incorporates two improved CA modules, positioned after the two Conv modules. Since the CA modules enhance the representational power of the feature maps, and the feature maps generated by the two Conv modules have different scales and depths, the two CA modules can act on the input image's feature maps at different scales, improving the effectiveness of the spatial structure in subsequent networks. In the Head section, a CoT module is added to each of the three Detect modules. CoT uses the fusion of static and dynamic context representations as its output, more fully utilizing the contextual information of the feature maps at different scales and improving classification accuracy.

[0049] The first aspect of this embodiment provides a method for infrared image target recognition of power transmission and transformation equipment based on edge computing, the method comprising:

[0050] S1, acquire the infrared image of the power transmission and transformation equipment to be identified, and perform preprocessing to scale it into a first image of a fixed size;

[0051] Specifically, this is achieved by resizing the image to a fixed size that fits the recognition model, such as 640×480.

[0052] S2, the first image is identified based on the target recognition network model deployed in the edge computing node.

[0053] Specifically, in this embodiment, the predetermined target recognition network model is based on the YOLOv7 network framework, including a backbone network and a head. The backbone network includes two CA modules, which are located after two convolutional modules (Conv). Each of the three Detect modules in the head incorporates a CoT module, which fuses the feature maps of static and dynamic context representations as the output. CA stands for CoordinateAttention, and CoT stands for Contextual Transformer.

[0054] The model generated by the described method will be deployed on edge computing nodes. Edge computing refers to analyzing data collected from the terminal directly on the local device or network near where the data was generated. The analysis refers to using the model to predict various power transmission and transformation equipment in the input power transmission and transformation infrared image. By deploying the trained model on edge computing nodes, low-latency, low-cost, and easily scalable local power transmission and transformation equipment identification can be achieved.

[0055] (a) CA module

[0056] Preferably, the CA module is used to perform one-dimensional horizontal global pooling and one-dimensional vertical global pooling on the input feature information, and then map the input to the range (0, 1) based on channel addition, two-dimensional convolution, normalization and activation operations so that the Re-weight module can assign weights to the residual connection information.

[0057] In this embodiment, one-dimensional horizontal global pooling and one-dimensional vertical global pooling are operations on the input, and the residual connection information is only used for the final weight allocation. Channel addition, two-dimensional convolution, normalization, and activation operations aim to complete feature selection through an attention mechanism. The Sigmoid layer completes the final mapping.

[0058] Specifically, regarding the network's structural details, the improved CA module... Figure 1 In Chinese, CA is used to represent this. The structure of a CA module is as follows: Figure 2 As shown.

[0059] like Figure 2 As shown, X Avg Pool and Y Avg Pool represent one-dimensional horizontal global pooling and one-dimensional vertical global pooling, respectively. These two pooling layers are used because global pooling is typically used for channel attention, globally encoding spatial information. However, it compresses global spatial information into the channel descriptor, making it difficult to preserve location information, which is crucial for the identification of power transmission and transformation equipment. To encourage attention blocks to capture long-term interactions spatially using precise location information, global pooling is decomposed into a pair of one-dimensional feature encoding operations, namely X Avg Pool and Y Avg Pool.

[0060] Specifically, for input X, the pooling kernel size used is (H, 1) or (1, W), encoding each channel along the horizontal and vertical coordinates respectively. Therefore, the output of the c-th channel at height h can be expressed as:

[0061]

[0062] Similarly, the output of the c-th channel with width w can also be written as:

[0063]

[0064] in and x represents the output of the c-th channel. c This represents the c-th channel of the input X. The two transformations described above aggregate features along two spatial directions, generating a pair of feature maps with direction-aware capabilities.

[0065] Concat adds the number of channels, Conv2d is a 2D convolution, BatchNorm+Non-linear is normalization plus an activation function, and Sigmoid maps the output to (0, 1) to facilitate re-weighting of residual connection information. Compared to the original CA module, the improved CA module adds two residual connections, avoiding information loss in the Concat and BatchNorm layers. The Add operation is a simple pixel stacking, increasing the amount of information describing the image features, but the dimension describing the image itself does not increase; only the amount of information in each dimension increases. This is obviously beneficial for the final image classification.

[0066] (b) CoT module

[0067] In this embodiment, the CoT module is used to map the static context representation of input data X to K. 1 And based on the contextualized attention matrix A, a new feature map K is generated. 2 ; Set the static context K 1 and dynamic context K 2 The output of the CoT module is generated through fusion using an attention mechanism.

[0068] The added CoT module is in Figure 1 In Chinese, it is represented by CoT. The structure of a CoT module is as follows: Figure 3 As shown.

[0069] like Figure 3 As shown, * denotes a local matrix multiplication operation. From a process perspective, assuming the input two-dimensional mapping is X, keys(K), queries(Q), and values(V) are defined as follows:

[0070] K = X (3)

[0071] Q = X (4)

[0072] V = XW v (5)

[0073] Among them, W v It's an embedding matrix. `keys`, `queries`, and `values` are elements of the Key, Query, and Value, respectively. From... Figure 3 From a process perspective, KeyMap maps the representation of each key to K using a k×k group convolution. 1 K 1 This refers to the static context representation of input X, after which K... 1 A concat operation is performed with the query to sum the channel counts. Then, the attention matrix is ​​implemented:

[0074] A = [K] 1 ,Q]W θ W δ (6)

[0075] Among them, K 1 It is a mapping of keys, W θ It is a 1×1 convolution with ReLU activation, while W δIt is a 1×1 convolution without an activation function, and A is the output attention matrix. In other words, for each head of multi-head attention, the local attention matrix at each spatial location of A is learned based on query features and contextualized key features, rather than a simple query-key mapping. This method leverages the mined static context K 1 Under the guidance of [the relevant authority], self-attention learning was enhanced. Then, based on the contextualized attention matrix A, a new feature map K was generated. 2 The following steps were performed to obtain the following result:

[0076] K 2 =V*A (7)

[0077] It can be seen that feature map K 2 It captures the dynamic feature interactions between inputs, K 2 This is referred to as the dynamic context representation of the input. Finally, the output of the CoT module is the static context K. 1 and dynamic context K 2 It is generated through the fusion of attention mechanisms.

[0078] (c) YOLOv7 built-in modules

[0079] In this embodiment, YOLOv7's built-in modules include ELAN, ELAN-H, MPConv, SPPCSPC, and RepConv. ELAN consists of multiple convolutional layers. The input and output feature sizes remain constant, but the number of channels changes after each convolution, with the final convolution outputting the desired number of channels. It uses expansion, random grouping, and cardinality merging to continuously enhance the network's learning ability without disrupting the original gradient path.

[0080] In terms of architecture, ELAN-H only changes the architecture within the computational blocks, while the architecture of the transition layers remains completely unchanged. This strategy utilizes group convolutions to expand the channels and cardinality of the computational blocks. Furthermore, it uses the same set of parameters and cross-channel integration for all computational blocks within a single computational layer. Besides retaining the traditional ELAN design architecture, ELAN-H can also guide different sets of computational blocks to learn more diverse features. ELAN-H also consists of multiple convolutional layers, and its operation is largely the same as ELAN, except that the final concatenation takes into account the results of the previous two convolutions; concatenation adds the number of channels. Its input and output feature sizes remain unchanged after the last convolution. Similarly, its number of channels is halved after two convolutions, and the final number of channels is twice the number of input channels. Its function is similar to ELAN.

[0081] The MPConv module has the same number of input and output channels, but the output size is half the input size. The upper part uses MaxPool to halve the size, and then uses convolution to halve the number of channels. The lower part first uses convolution to halve the number of channels, and then uses a convolution with a stride of 2 to halve the size. Finally, the upper and lower parts are merged using cat to obtain the output. MaxPool is the maximum pooling operation.

[0082] The SPPCSPC network structure mainly consists of convolutional layers and a MaxPool. The output layer of the entire SPPCSPC layer has _out_c output channels, and during training, a hidden layer channel, _hidden_c, is calculated.

[0083] int(2*e*out_c) is used to expand the amount of information. Generally, e = 0.5 is taken, then hidden_c = out_c.

[0084] RepConv uses different architectures for training and inference, and employs model reparameterization. Reparameterization can be viewed as an ensemble technique, categorized into module-level ensemble and model-level ensemble. For model-level reparameterization, two common practices exist to obtain the final inference model: One is to train multiple identical models using different training data and then average the weights of these models. The other method is to perform a weighted average of the model weights across different iterations. Module-level reparameterization divides a module into multiple identical or different branches during training and integrates these branches into a completely equivalent module during inference. Specifically, in the network structure, training consists of a 3x3 convolutional branch and a 1x1 convolutional branch. If the number of input and output channels and their sizes are the same, a branch with only a batch normalization (BN) layer is added, and the three branches are summed for the output. During inference, to improve efficiency, the parameters of the branches are reparameterized onto the main branch, and the 3x3 main branch convolution is used for the output.

[0085] S3, obtain the recognition result of the first image; the recognition result is a target prediction box generated by a predetermined target recognition network model.

[0086] Preferably, the method further includes a training process for a predetermined target recognition network model; the training process includes: generating a training set and a test set using infrared images of several power transmission and transformation equipment; selecting accuracy, recall, and mAP as performance indicators, and training and optimizing the model based on preset recognition accuracy and the ratio of IOU between predicted boxes and true boxes, so that the overlap between predicted boxes and actual equipment bounding boxes is high.

[0087] Here, the metrics are calculated from the trained model. Accuracy refers to the proportion of true positive classes out of all correctly predicted positive classes. Recall refers to the proportion of true positive classes out of all correctly predicted positive classes. IOU stands for Intersection over Union, representing the degree of overlap between the ground truth bounding boxes and the predicted bounding boxes. The number of training epochs is generally chosen based on the model's convergence; training can usually be terminated when the changes in the metrics are not significant.

[0088] The experiment mainly consisted of three steps: dataset creation, dataset training, and model testing. Specifically, dataset creation involved labeling transmission and transformation equipment such as insulators, conductors, hardware, and transformer bushings in infrared images, and then dividing the labeled dataset into training and validation sets proportionally. The training part involved feeding the dataset into the CACoT-YOLOv7 network, setting the target types to be identified, and configuring the model parameters. After training, the best-performing model was used for testing. The final testing used unlabeled images, and the model's accuracy was determined by comparing the predicted bounding boxes with the actual bounding boxes.

[0089] Our experiment used 5821 infrared images, with 5182 images used as the training set and 639 images used as the test set. During training, our epoch was set to 170, and the performance metrics are as follows: Figure 4 As shown, its accuracy is between 86% and 87%. mAP@0.5 represents the average mAP for all classes with an IOU (Intersection over Union) threshold greater than 0.5, where IOU is the ratio of the intersection of the predicted bounding box and the true bounding box to the union of the predicted bounding box and the true bounding box. mAP@0.5 eventually stabilizes at around 0.77. AP represents the average precision, and mAP represents the average AP across all classes.

[0090] The final model's predicted image is as follows Figure 5 As shown, the model trained by the CACoT-YOLOv7 network can effectively identify power transmission and transformation equipment of various sizes, and the predicted boxes generated by the model have a high degree of overlap with the actual equipment's labeled boxes.

[0091] The actual application of the model will be deployed on edge computing nodes. Edge computing provides edge intelligence services at the network edge, close to the source of objects or data, through a distributed open platform that integrates core capabilities of networking, computing, storage, and applications. The edge computing architecture is as follows: Figure 6As shown, the terminal node is the device that captures infrared images, primarily responsible for collecting and uploading raw data. Edge computing nodes provide basic service responses by strategically deploying and allocating the computing and storage capabilities of network edge nodes. Network nodes are responsible for uploading useful data processed by the edge computing nodes to cloud computing nodes for analysis. Cloud computing nodes permanently store the data reported from the edge computing layer. Meanwhile, analysis tasks and tasks involving the processing of comprehensive global information that edge computing nodes cannot handle still need to be completed at the cloud computing nodes. Furthermore, cloud computing nodes can dynamically adjust the deployment strategies and algorithms of the edge computing layer based on network resource distribution.

[0092] In this embodiment, the infrared image target recognition of power transmission and transformation equipment based on CACoT-YOLOv7 can accurately identify various types of power transmission and transformation equipment. Compared with traditional methods and existing deep learning-based network models, it improves the network's representation ability and classification accuracy by adding improved CA and CoT modules to the YOLOv7 network. The model trained using this network can effectively complete the infrared image target recognition task of power transmission and transformation equipment, thereby enabling real-time monitoring of the equipment's status. This is beneficial for improving the operational reliability and resource utilization of power transmission and transformation equipment, and is also significant for reducing maintenance costs.

[0093] Furthermore, a second aspect of this embodiment provides an electronic device comprising: one or more processors, and a memory for storing one or more computer programs; characterized in that the computer programs are configured to be executed by the one or more processors, and the programs include steps for performing the edge computing-based infrared image target recognition method for power transmission and transformation equipment as described in the first aspect.

[0094] Furthermore, a third aspect of this embodiment provides a storage medium storing a computer program; the program is loaded and executed by a processor to implement the steps of the edge computing-based infrared image target recognition method for power transmission and transformation equipment as described in the first aspect.

[0095] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0097] The units described as separate components may or may not be physically separate. As will be appreciated by those skilled in the art, the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0098] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0099] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or grid device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for infrared image target recognition of power transmission and transformation equipment based on edge computing, characterized in that, The method includes: Acquire infrared images of the power transmission and transformation equipment to be identified, and perform preprocessing to scale them into a first image of a fixed size; The first image is identified based on a target recognition network model deployed in edge computing nodes. The target recognition network model is based on the YOLOv7 network framework and includes a backbone network and a head. The backbone network includes two CA modules, which are located after two convolutional modules (Conv). Each of the three Detect modules in the head includes a CoT module, which fuses the feature maps of static and dynamic context representations as the output. The CA module is used to perform one-dimensional horizontal global pooling and one-dimensional vertical global pooling on the input feature information, and then map the input to the range (0, 1) based on channel addition, two-dimensional convolution, normalization and activation operations so that the Re-weight module can assign weights to the residual connection information. The CoT module is used to map the static context representation of the input data X to... And based on the contextualized attention matrix A, a new feature map is generated. ; Set static context and dynamic context The output of the CoT module is generated through fusion via an attention mechanism; Obtain the recognition result of the first image; the recognition result is the target prediction box generated by the set target recognition network model.

2. The infrared image target recognition method for power transmission and transformation equipment based on edge computing according to claim 1, characterized in that, The predetermined target recognition network model also includes YOLOv7 built-in modules; the YOLOv7 built-in modules include ELAN, ELAN-H, MPConv, SPPCSPC, and RepConv; ELAN consists of multiple convolutional layers, the input and output feature sizes remain unchanged, the number of channels changes after passing through two convolutions, and the output after the last convolution is the required number of channels; ELAN-H also consists of multiple convolutional layers, and its operation takes into account the results of the previous two convolutions when concatenating at the end of ELAN; The MPConv module has the same number of input and output channels, but the output size is half the input size. The upper part is halved by MaxPool and then halved by convolution. The lower part is halved by convolution and then halved by a convolution with a stride of 2. The upper and lower parts are then merged by cat to obtain the output. The SPPCSPC network structure mainly consists of convolutional layers and MaxPool; the output layer channel of the entire SPPCSPC layer is out_c. During training, a hidden layer channel hidden_c = int(2*e*out_c) is calculated to expand the information content. Taking e = 0.5, then hidden_c = out_c.

3. The infrared image target recognition method for power transmission and transformation equipment based on edge computing according to claim 2, characterized in that, The RepConv uses different structures during training and inference, and employs model reparameterization techniques. During training, RepConv consists of a 3x3 convolutional branch and a 1x1 convolutional branch. If the number of input and output channels and their sizes are the same, a branch with only a BN layer is added, and the three branches are summed for output. During inference, to improve efficiency, the parameters of the branches are reparameterized onto the main branch, and the 3x3 main branch convolution is used for output.

4. The infrared image target recognition method for power transmission and transformation equipment based on edge computing according to claim 3, characterized in that, The method also includes a training process for a predetermined target recognition network model; The training process includes: generating training and testing sets using infrared images of several power transmission and transformation equipment; Set performance metrics such as accuracy, recall, and IOU. Based on preset recognition accuracy and the ratio of IOU between predicted bounding boxes and true bounding boxes, train and optimize the model to achieve a high degree of overlap between predicted bounding boxes and actual labeled bounding boxes on the device.

5. An electronic device, the electronic device comprising: One or more processors, a memory for storing one or more computer programs; characterized in that the computer programs are configured to be executed by the one or more processors, the programs including steps for performing the edge computing-based infrared image target recognition method for power transmission and transformation equipment as described in any one of claims 1-4.

6. A storage medium storing a computer program; the program being loaded and executed by a processor to implement the steps of the edge computing-based infrared image target recognition method for power transmission and transformation equipment as described in any one of claims 1-4.

Images