Method, device and equipment for identifying power equipment in infrared image

Abstract

The invention discloses a method, a device and equipment for identifying power equipment in an infrared image. The method comprises the following steps: acquiring a collected infrared image of a power grid; the infrared image is input into a preset fusion reasoning model, equipment information of all the power equipment in the infrared image is obtained, and the fusion reasoning model comprises a RetinaNet sub-model for extracting temperature features after positive and negative samples in the infrared image are subjected to balance processing, an OfficientDet sub-model for extracting the temperature features of main power equipment in the power equipment, and a RetinaNet sub-model for extracting the temperature features of the main power equipment in the power equipment; the Scaled-YOLOv4 submodel is used for extracting the temperature characteristics of the local heating element in the power equipment; and screening target information of target equipment located in the center of the infrared image from the equipment information. That is, through a collaborative mechanism of multi-model heterogeneous fusion and central target focusing, the accuracy of identifying the target power equipment from the infrared image in a dense and similar equipment scene is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of the power industry, in particular to a method and device for identifying power equipment in an infrared image and equipment thereof. BACKGROUND

[0002] With the popularization of infrared and partial discharge thermal imaging technology, the research and development technology is increasingly improved, and the infrared / partial discharge thermal imaging technology has also continuously entered the civilian field, especially the power industry.

[0003] At present, the infrared and partial discharge thermal imaging technology is mainly used for real-time detection of power equipment in the power industry. Since the real-time detection is a non-destructive and non-contact detection method, the application of infrared and partial discharge thermal imaging technology in the power industry can improve the detection efficiency of power equipment. However, with the development of power Internet of Things and digitization, the structure of power equipment is also more complex, and there are devices with similar size and structure in power equipment. Therefore, the accuracy of identifying power equipment from infrared images is low.

[0004] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide a method and device for identifying power equipment in an infrared image and equipment thereof, aiming to solve the technical problem of low accuracy of identifying power equipment from infrared images.

[0006] To achieve the above purpose, the present application provides a method for identifying power equipment in an infrared image, which comprises: acquiring an infrared image of a power grid collected; inputting the infrared image into a preset fusion inference model to obtain device information of all power equipment in the infrared image, wherein the fusion inference model comprises a RetinaNet sub-model for extracting temperature features of positive and negative samples after balancing processing in the infrared image, an EfficientDet sub-model for extracting temperature features of main power equipment in the power equipment, and a Scaled-YOLOv4 sub-model for extracting temperature features of local heating elements in the power equipment; screening target information of a target device located at the center of the infrared image from the device information.

[0007] In an embodiment, the step of inputting the infrared image into a preset fusion inference model to obtain device information of all power equipment in the infrared image comprises: inputting the infrared image into a preset fusion inference model to pre-process the infrared image and obtain a temperature matrix; extracting first temperature features of all power equipment in the temperature matrix based on the RetinaNet sub-model after the positive and negative sample balancing processing of the temperature matrix; extracting second temperature features based on the EfficientDet sub-model for the backbone power equipment in the temperature matrix; extracting third temperature features based on the Scaled-YOLOv4 sub-model for the local heating elements in the temperature matrix; performing splicing classification processing on the first temperature features, the second temperature features and the third temperature features to obtain device information of all power equipment in the infrared image of the device.

[0008] In an embodiment, before the step of inputting the infrared image into a preset fusion inference model and obtaining a temperature matrix by preprocessing the infrared image, the method further comprises: obtaining a historical infrared image of the power grid; training an initial fusion model by taking the historical infrared image as sample data, obtaining a preliminary training fusion model, and testing the accuracy of each sub-model in the preliminary training fusion model; establishing a new fully connected layer in each of the sub-models; weighting the new fully connected layer based on the test accuracy to obtain an initialized fully connected layer; freezing the trained convolutional layer parameters in the preliminary training fusion model, and obtaining a fusion inference model after feature extraction training of the initialized fully connected layer.

[0009] In an embodiment, before the step of extracting first temperature features of all power equipment in the temperature matrix based on the RetinaNet sub-model after the positive and negative sample balancing processing of the temperature matrix, the method further comprises: performing dimension conversion on the infrared image based on the fusion inference model to obtain three-dimensional temperature data; performing data stacking processing on the three-dimensional temperature data to obtain three-dimensional multi-channel data; performing normalization processing on the three-dimensional multi-channel data; performing dimension expansion processing on the normalized three-dimensional multi-channel data to obtain a temperature matrix.

[0010] In an embodiment, the device information includes device type, device location and classification confidence. The step of performing splicing classification processing on the first temperature features, the second temperature features and the third temperature features to obtain device information of all power equipment in the infrared image of the device comprises: weighting processing is performed on the first temperature feature, the second temperature feature and the third temperature feature; The first temperature feature, the second temperature feature and the third temperature feature after the weighting processing are size-unified spliced to obtain a spliced feature; Based on the spliced feature, all power equipment in the infrared image is classified to determine the device type, device location and classification confidence of all power equipment.

[0011] In an embodiment, if the identification of the power equipment in the power grid is re-identification, after the step of inputting the infrared image into a preset fusion inference model to pre-process the infrared image to obtain a temperature matrix, the method further comprises: The temperature matrix and each saved historical matrix in a preset feature library are subjected to Euclidean distance calculation to obtain the Euclidean distance between the temperature matrix and each historical matrix; Based on the Euclidean distance, the similarity between the power equipment and the recorded equipment corresponding to each historical matrix is determined; The equipment with the highest similarity among the recorded equipment is selected as the target equipment in the center of the infrared image.

[0012] In an embodiment, after the step of obtaining the infrared image of the power grid collected, the method further comprises: The infrared image is subjected to preliminary equipment identification to obtain an identification result; If the identification result is that the power equipment is a one-time equipment, the step of inputting the infrared image into a preset fusion inference model is executed to obtain the device information of all power equipment in the infrared image.

[0013] In an embodiment, the step of identifying the infrared image to obtain an identification result comprises: The infrared image is subjected to dimension conversion processing to obtain three-dimensional single-channel data; The three-dimensional single-channel data is input into a preset classification model to obtain an identification result.

[0014] In addition, to achieve the above-mentioned purpose, the present application further provides a power equipment identification device in an infrared image, which comprises: An acquisition module is configured to acquire an infrared image of a power grid collected; The inference module is configured to input the infrared image into a preset fusion inference model to obtain device information of all power equipment in the infrared image, wherein the fusion inference model comprises a RetinaNet sub-model configured to extract temperature features of positive and negative samples after balance processing, an EfficientDet sub-model configured to extract temperature features of main power equipment in the power equipment, and a Scaled-YOLOv4 sub-model configured to extract temperature features of local heating elements in the power equipment. The screening module is configured to screen target information of target equipment located at the center of the infrared image from the device information.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a device for identifying power equipment in an infrared image, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the method for identifying power equipment in an infrared image.

[0016] The one or more technical solutions provided by the present application have at least the following technical effects: By constructing the fusion inference model comprising multiple special sub-models, the infrared image obtained is identified to obtain power equipment, and the identification capability of power equipment in a complex scene is improved, wherein the RetinaNet sub-model in the fusion inference model extracts global temperature distribution features on the basis of positive and negative sample balance processing, and the missing detection problem caused by background interference or small target sparseness is alleviated; the EfficientDet sub-model in the fusion inference model focuses on the structure and thermal state features of the main power equipment, and enhances the positioning robustness of the key equipment; the Scaled-YOLOv4 sub-model specially captures the subtle temperature rise mode of the local heating element, and improves the recognition accuracy of the fine-grained component, and the three work together to extract complementary temperature semantic information in the infrared image of the power grid from three levels of overall layout, main structure, and local hot spot, form a multi-level and multi-granularity joint representation of the power equipment, and thus accurately obtain the device information of each power equipment in the infrared image. Since the power equipment is usually detected by shooting the target equipment to be detected in the middle of the image through the infrared image, in order to accurately identify the power equipment, it is also necessary to screen the target information located at the center from the infrared image according to the device information. That is, through the synergistic mechanism of multi-model heterogeneous fusion and center target focusing, the accuracy of identifying the target power equipment from the infrared image in a dense and similar equipment scene is improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0019] Figure 1 The flowchart provided by the embodiment of the present application for identifying power equipment in an infrared image; Figure 2 The flowchart provided by the embodiment of the present application for identifying power equipment in an infrared image; Figure 3 The flowchart provided by the embodiment of the present application for identifying power equipment in an infrared image; Figure 4 The module structure diagram of the identification device for power equipment in an infrared image according to the embodiment of the present application; Figure 5 The device structure diagram of the hardware running environment related to the method for identifying power equipment in an infrared image according to the embodiment of the present application.

[0020] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0022] In order to better understand the technical solutions of the present application, the following will be described in detail with reference to the drawings and specific embodiments.

[0023] It should be noted that the execution subject of the present embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a power dispatching platform, etc. capable of realizing the above functions. The present embodiment and the following embodiments will be described taking the power dispatching platform as an example.

[0024] Based on this, the present embodiment provides a method for identifying power equipment in an infrared image, which will be described in detail with reference to Figure 1 , Figure 1 The flowchart of the embodiment of the present application for identifying power equipment in an infrared image.

[0025] In the present embodiment, the method for identifying power equipment in an infrared image comprises steps S10-S30: Step S10, acquiring the infrared image of the power grid collected; It should be noted that the power grid is a power system network composed of power generation, power transmission, power transformation, power distribution and the like. The infrared image is two-dimensional image data reflecting the surface temperature distribution of the power equipment in the power grid collected by an infrared thermal imaging device; wherein the gray value or color value of each pixel point in the image corresponds to the radiation temperature of the measured equipment at the position, which is used to represent the thermal field characteristics of the equipment in the running state.

[0026] In a specific implementation, the infrared thermal imager carried by a handheld device or a drone can be used to inspect the substation or power transmission line, and real-time collection of infrared images containing multiple power equipment can be performed; the image is stored in the form of a digital matrix, each element representing the temperature quantization value of the corresponding spatial position, and is transmitted to the background processing terminal through a wired or wireless communication link.

[0027] It can be understood that the infrared image can also be accompanied by metadata information such as shooting time, environmental temperature, equipment rated load, distance parameter and lens focal length, which is encoded as a context vector and input into the early layers of the fusion inference model to assist the fusion inference model in temperature normalization compensation (for example: the same temperature rise corresponds to different fault levels under different environmental temperatures), thereby improving the physical consistency and diagnostic accuracy of the identification result.

[0028] Step S20, inputting the infrared image into a preset fusion inference model to obtain device information of all power equipment in the infrared image, the fusion inference model including a RetinaNet sub-model for extracting temperature features of positive and negative samples after balance processing in the infrared image, an EfficientDet sub-model for extracting temperature features of a main power equipment in the power equipment, and a Scaled-YOLOv4 sub-model for extracting temperature features of a local heating element in the power equipment; It should be noted that the fusion inference model is a composite neural network architecture integrating multiple deep learning sub-models, which is used for multi-granularity and multi-task joint analysis of infrared images. The RetinaNet sub-model is a sub-network constructed based on the RetinaNet target detection framework, which is used to handle the problem of extremely unbalanced positive and negative samples caused by the fact that the background area is much larger than the device area in the infrared image. The focal loss mechanism is used to strengthen the learning ability of the sparse power device target, and the global temperature features of all power devices in the whole image range are extracted. The EfficientDet sub-model is a sub-network constructed based on the EfficientDet target detection architecture, which focuses on identifying and extracting the structured temperature features of the backbone power devices (such as large-size devices such as transformers, circuit breakers, disconnectors, or current transformers) in the power grid. The Scaled-YOLOv4 sub-model is an improved sub-network based on Scaled-YOLOv4, which is used to detect local heating elements (such as small-size components prone to overheating, such as terminal connectors, bushings, contacts, and connecting bolts) in the infrared image. The Scaled-YOLOv4 sub-model has high-resolution feature map processing capability, which can effectively capture weak but critical local temperature rise anomalies. The device information is the structured data output by the fusion inference model, which at least includes the type of power device, the position coordinates of the power device in the image, and the classification confidence.

[0029] It can be understood that by introducing three sub-models with complementary advantages, full-scene coverage of power devices is achieved, that is, small target missed detection and sample imbalance problems are solved by the RetinaNet sub-model, the stability of large-size device identification is ensured by the EfficientDet sub-model, and the sensitivity to small-size devices is improved by the Scaled-YOLOv4 sub-model. The three work together to improve the recognition performance of the fusion inference model in complex infrared scenes.

[0030] Optionally, the operation between the RetinaNet sub-model, the EfficientDet sub-model, and the Scaled-YOLOv4 sub-model in the fusion inference model can adopt a feature pyramid model interaction mechanism; that is, the low-layer high-resolution feature map output by the RetinaNet sub-model is upsampled and input as an attention guide signal to the corresponding level of the Scaled-YOLOv4 sub-model, to strengthen the attention of the Scaled-YOLOv4 sub-model to the tiny heating area; the high-layer semantic features output by the EfficientDet sub-model are broadcast to the classification head of the RetinaNet sub-model to suppress false positives of non-large-size devices.

[0031] Optionally, since the learnable gating unit is composed of a lightweight MLP, the contribution of each sub-model can be automatically adjusted according to the complexity of the input image, and therefore, the features can also be dynamically weighted before being spliced through the learnable gating unit.

[0032] Optionally, the fusion inference model can adopt a knowledge distillation compression strategy when deployed, taking the complete fusion model as a teacher network, training a lightweight student network for deployment on resource-constrained devices such as unmanned aerial vehicles or handheld terminals, and realizing cloud-edge collaborative inference.

[0033] Step S30: filtering target information of a target device located at the center of the infrared image from the device information.

[0034] It should be noted that the target device is a power device that needs to be analyzed or alarmed in the current inspection task. The target information is the complete identification result corresponding to the target device filtered from the device information. The image center is a target focus area dynamically adjusted based on the device density heat map, and is not a fixed geometric center.

[0035] It can be understood that, in the actual power inspection process, the operator usually places the device to be inspected in the center of the field of view to ensure clear imaging and accurate temperature measurement; therefore, taking the image center device as the target device is more in line with the human-machine operation habit, can automatically focus on the current object of interest, and avoids misjudgment or redundant alarm on the background device, thereby greatly simplifying the subsequent diagnosis process and improving the human-machine cooperation efficiency and automation level.

[0036] In a specific implementation, the device density heat map can be generated by using the rough device position output by the RetinaNet sub-model; then the local maximum value point is found by Gaussian kernel smoothing, to determine the most likely aligned device cluster center in the current image; finally, the target region is determined based on the dynamic center, to determine the target device from the target region, thereby obtaining the target information.

[0037] Optionally, in the continuous frame inspection scene, a short-term device trajectory cache can also be maintained, that is, if the spatial displacement between the current frame center device and the target device of the previous frame is less than a threshold, it is determined that the same device is detected, a time sequence consistency check is triggered, and when the same type of device is identified with high confidence in two consecutive frames, the final target information is output, thereby suppressing single frame false detection.

[0038] The embodiment provides a method for identifying power equipment in an infrared image. The method comprises constructing a fusion inference model comprising a plurality of special sub-models, identifying power equipment in the obtained infrared image, and improving the identification capability of the power equipment in a complex scene. The RetinaNet sub-model in the fusion inference model extracts global temperature distribution features on the basis of positive and negative sample balancing processing, thereby relieving the missed detection problem caused by background interference or small target sparseness. The EfficientDet sub-model in the fusion inference model focuses on the structure and thermal state features of the main power equipment, thereby enhancing the positioning robustness of the key equipment. The Scaled-YOLOv4 sub-model specially captures the subtle temperature rise mode of the local heating element, thereby improving the identification accuracy of the fine-grained components. The three sub-models work together to extract complementary temperature semantic information of the power grid in the infrared image from three levels of overall layout, main structure, and local hot spot, form a multi-level and multi-granularity joint representation of the power equipment, and accurately obtain the equipment information of each power equipment in the infrared image. Since the power equipment is usually detected by shooting the target equipment to be detected in the middle of the image, in order to accurately identify the power equipment, the target information located in the center needs to be screened from the infrared image according to the equipment information. That is, through the synergistic mechanism of multi-model heterogeneous fusion and center target focusing, the accuracy of identifying the target power equipment from the infrared image in a dense and similar equipment scene is improved.

[0039] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above-mentioned first embodiment can be referred to the above introduction, and will not be described in detail. On this basis, please refer to Figure 2 , step S20 further comprises steps S01-S05: Step S01, inputting the infrared image into a preset fusion inference model, pre-processing the infrared image to obtain a temperature matrix; Step S02, based on the RetinaNet sub-model, performing positive and negative sample balancing processing on the temperature matrix, and extracting first temperature features of all power equipment in the temperature matrix; Step S03, based on the EfficientDet sub-model, performing temperature feature extraction on the main power equipment in the temperature matrix to obtain second temperature features; Step S04, based on the Scaled-YOLOv4 sub-model, performing temperature feature extraction on the local heating element in the temperature matrix to obtain third temperature features; Step S05, performing splicing and classification processing on the first temperature features, the second temperature features and the third temperature features, and obtaining equipment information of all power equipment in the infrared image.

[0040] It should be noted that the preprocessing is a series of data transformation operations on the original infrared image, including but not limited to dimension adjustment, channel stacking, normalization and matrix reconstruction. The temperature matrix is a multi-channel numerical matrix formed after preprocessing, which has a unified data format and dimension, and each element represents the temperature response value at the spatial position. The temperature feature is a high-dimensional feature vector or feature map representing the thermal response of power equipment at different scales and semantic levels. The device information includes device type, device location and classification confidence.

[0041] It can be understood that, since the proportion of equipment in the power grid infrared image is usually less than 5%, and the traditional cross-entropy loss is easy to cause the model to be biased towards predicting the background; since RetinaNet can improve the recall ability of sparse power equipment through the focal loss mechanism, it is especially suitable for substation panoramic inspection or long-distance shooting scenes of power transmission lines, therefore, using the RetinaNet sub-model to balance the positive and negative samples of the infrared image, and then obtaining the global temperature feature, can effectively avoid missing detection and ensure the integrity of the identification.

[0042] It can be understood that, through multi-source feature fusion instead of simple voting or cascade, the present application realizes the triple protection of global coverage, accurate backbone and local sensitivity; and further uses a dynamic weighting mechanism to further ensure adaptive allocation of model resources in different scenes, avoiding redundant calculation and preventing loss of key features, thereby significantly improving the accuracy and robustness of identification as a whole.

[0043] It can be understood that, since local overheating is often an early sign of equipment failure, but due to the small size and limited temperature rise of such power equipment, it is easily ignored by conventional detection models, and the Scaled-YOLOv4 has high resolution perception ability, using the Scaled-YOLOv4 sub-model to extract the temperature feature of the small-size local heating element in the infrared image can improve the detection rate of small-size power equipment.

[0044] Further, before step S01, the method for identifying power equipment in an infrared image further comprises: acquiring a historical infrared image of the power grid; training an initial fusion model using the historical infrared image as sample data, obtaining a preliminary training fusion model, and testing the accuracy of each sub-model in the preliminary training fusion model; establishing a new fully connected layer in each of the sub-models; weighting the new fully connected layer based on the test accuracy of each sub-model to obtain an initialized fully connected layer; freezing the trained convolutional layer parameters in the preliminary training fusion model, and training the initialized fully connected layer for feature extraction to obtain a fusion inference model.

[0045] It should be noted that the historical infrared image is a collection of infrared images of power equipment in the power grid recorded by an infrared thermal imaging device under different times, different environmental conditions and different operating loads. The initial fusion model is a composite neural network architecture composed of RetinaNet sub-model, EfficientDet sub-model and Scaled-YOLOv4 sub-model without task fine-tuning. The initial training fusion model is an intermediate model obtained by training the initial fusion model end-to-end or in stages using the historical infrared image as a training sample. The test accuracy is a quantitative indicator obtained by evaluating the performance of each sub-model in the initial training fusion model on an independent validation set. The new fully connected layer is a fully connected neural network layer with the same structure but randomly initialized parameters added after the original classification / regression head of each sub-model. The initialized fully connected layer is a set of fully connected layers with differentiated initial parameters formed after weighted processing.

[0046] It can be understood that freezing the trained convolutional layer parameters in the initial training fusion model can be understood as setting the weights of all convolutional layers (including the backbone network and the feature pyramid) in the initial training fusion model to a non-trainable state during model fine-tuning, and only allowing the parameters of the new fully connected layer to be updated through backpropagation.

[0047] It can be understood that fully training the initial training model using historical infrared images can ensure the underlying feature extraction capability, and through the new fully connected layer and weighted initialization, a learnable and interpretable fusion interface can be constructed. Then, using the transfer learning paradigm of freezing convolutional layers and fine-tuning fully connected layers, both performance and efficiency can be considered, thus improving the recognition accuracy and robustness of the fusion model.

[0048] It can be understood that by freezing the trained convolutional layers, not only the general infrared feature extraction capability learned by the initial training fusion model from massive historical data is preserved, but also the performance degradation caused by overfitting during small sample fine-tuning is avoided. At the same time, only the lightweight fully connected layer is trained, so that the entire fusion inference model can be efficiently deployed on resource-constrained edge devices. Therefore, the best balance between model performance, training efficiency and deployment feasibility is achieved.

[0049] It can be understood that by converting the test accuracy into the initial value of the physical parameters of the fully connected layer, the high-reliability model occupies a dominant position at the starting point of training, greatly accelerating convergence and improving the final fusion performance.

[0050] Understandably, by introducing a new fully connected layer with consistent structure, the three originally heterogeneous sub-models (RetinaNet, EfficientDet, and Scaled-YOLOv4) can output aligned prediction results in the same semantic space, laying a structural foundation for feature concatenation or decision fusion. At the same time, isolating the original detection head from the new output layer can also prevent the fine-tuning process from destroying the learned low-level feature representations.

[0051] Furthermore, prior to step S02, the method for identifying electrical equipment in infrared images further includes: Based on the fusion inference model, the infrared image is dimensionally transformed to obtain three-dimensional temperature data; The three-dimensional temperature data is stacked to obtain three-dimensional multi-channel data; The three-dimensional multi-channel data is normalized. The normalized three-dimensional multi-channel data is subjected to dimensional expansion processing to obtain the temperature matrix.

[0052] It should be noted that the three-dimensional tensor formed by transforming the original single-channel infrared image into a single temperature channel, denoted as H×W×1, is a multi-dimensional tensor with height, width, and a single temperature channel. Data stacking is the operation of stitching together multiple two-dimensional or three-dimensional data with the same spatial dimension according to the channel dimension to form a higher-dimensional multi-channel tensor. The three-dimensional multi-channel data is a numerical matrix with three dimensions and more than one channel, generated after data stacking, denoted as H×W×C (where C≥2).

[0053] Where H is the height, W is the width, and C is the number of channels.

[0054] It should be noted that dimensional expansion refers to increasing the number of channels or adjusting the spatial dimensions of normalized three-dimensional multi-channel data.

[0055] Understandably, by using dimensional expansion processing, seamless compatibility with existing efficient model architectures can be achieved without losing the original infrared information. At the same time, the use of intelligent expansion strategies such as reflection filling can also avoid artificial boundary artifacts introduced at the image edges, ensuring the recognition accuracy of edge devices.

[0056] In the specific implementation, the received one-dimensional temperature data is transformed into a 640x480x1 three-dimensional array; the data is stacked in the depth direction, transforming the 640x480x1 temperature data into a 640x480x3 three-dimensional array; the 640x480x3 three-dimensional array is normalized and expanded in dimension so that the structure of the temperature data is transformed into a temperature matrix in the form of 1x640x480x3.

[0057] Furthermore, step S05 also includes: The first temperature feature, the second temperature feature, and the third temperature feature are weighted; The first, second, and third temperature features after weighting are uniformly spliced ​​together to obtain the spliced ​​features. Based on the stitching features, all electrical devices in the infrared image are classified to determine the device type, location, and classification confidence level of each device.

[0058] It should be noted that the size uniform stitching is an operation performed after weighting, aligning the spatial resolution and stitching the channel dimensions of the three types of temperature features to form a single, continuous high-dimensional feature tensor. Equipment type is the specific category identifier of the power equipment, used to distinguish equipment with different functions and structures. Equipment location is the spatial positioning information of the power equipment in the infrared image. Classification confidence is a quantitative value of the model's reliability in predicting the equipment type and location.

[0059] It is understood that this embodiment transforms prediction uncertainty into dynamic weights, enabling the fusion process to automatically suppress the interference of low-confidence sub-models and strengthen the dominant role of high-reliability sub-models; thereby improving robustness in complex, noisy, or occluded scenarios and avoiding overall failure due to misjudgment by a single sub-model.

[0060] Understandably, due to the different feature pyramid levels and receptive field designs of RetinaNet, EfficientDet, and Scaled-YOLOv4, their output features have spatial alignment deviations. Direct splicing would introduce geometric misalignment noise. However, through deformable convolution and multi-scale alignment, sub-pixel-level feature registration can be achieved, ensuring that the spliced ​​features are spatially accurate and consistent. At the same time, cross-feature interaction layers enable the model to infer the relationship between local heating and the attribution of backbone equipment, thereby improving the accuracy of identifying complex faults.

[0061] Optionally, if the identification of electrical equipment in the power grid is a re-identification, after step S01, the method for identifying electrical equipment in the infrared image further includes: The temperature matrix is ​​compared with each of the historical matrices stored in the preset feature library using Euclidean distance calculation to obtain the Euclidean distance between the temperature matrix and each of the historical matrices. Based on the Euclidean distance, the similarity between the power equipment and the recording equipment corresponding to each of the historical matrices is determined; The device with the highest similarity among the various recording devices is selected as the target device at the center of the infrared image.

[0062] It should be noted that the preset feature library is a database containing multiple historical thermal feature samples of power equipment, built and stored before or during the deployment of the power dispatching platform. The historical matrix is ​​a temperature matrix extracted and saved from infrared images under normal or typical operating conditions, corresponding to a specific recording device. Euclidean distance is a metric for measuring the geometric distance between two matrices in a multi-dimensional vector space. Similarity is the degree of closeness in thermal distribution patterns between the current power equipment and historical record devices. Recording devices are the known power equipment entities corresponding to each historical matrix in the preset feature library.

[0063] Understandably, when the current central device is determined to be highly similar to a certain recording device, its ledger information can be automatically linked to assist maintenance personnel in quickly identifying the inspection target; if the similarity is lower than the threshold, a misalignment or device malfunction will be indicated, guiding the user to retake the photo, thereby improving the success rate of the inspection.

[0064] Understandably, while the fusion reasoning model can output device type and location, it cannot distinguish individual devices of the same type. The process of determining the target device with the highest similarity, the fusion reasoning model, temperature matrix preprocessing, and device information output are deeply coordinated to form an intelligent infrared analysis platform that integrates identification, confirmation, focusing, and diagnosis, which can make up for the identity blind spot of the pure detection model.

[0065] In practical implementation, for a fixed substation, the equipment in the substation is fixed. In this case, it is possible to save the type information and the feature vector extracted by the last fully connected layer of the classification model during the initial identification. When identifying the equipment type of the substation again, the feature vector of the equipment to be identified can be compared with the feature vector of all the equipment saved in the substation. The similarity between the two devices can be determined by calculating the Euclidean distance between each pair of feature vectors. The device with the highest similarity, i.e. the smallest Euclidean distance, can be identified as the same device.

[0066] Based on Embodiments 1 and 2 of this application, the same or similar content in Embodiment 3 of this application can be referred to the above description, and will not be repeated hereafter. Please refer to [the above description]. Figure 2 After step S10, the method for identifying electrical equipment in infrared images further includes steps S1-S2: Step S1: Perform preliminary device identification on the infrared image to obtain the identification result; Step S2: If the identification result indicates that the power equipment is a primary device, then the infrared image is input into a preset fusion inference model to obtain the device information of all power equipment in the infrared image.

[0067] It should be noted that primary equipment refers to high-voltage electrical equipment that directly participates in the production, transmission, distribution, and use of electrical energy.

[0068] Understandably, not all infrared images contain primary equipment that requires high-precision analysis; if computationally intensive fusion inference models are called for all images, it will result in a large amount of ineffective computing power consumption and reduce system throughput efficiency. Therefore, by introducing a low-overhead preliminary identification step, intelligent task filtering can be achieved, and high-precision analysis can be initiated when necessary, thereby optimizing the allocation of computing resources.

[0069] Understandably, primary equipment is the core asset of power grid security, and its status is directly related to the stable operation of the system, thus requiring high-precision, multi-model fusion-based in-depth analysis. Secondary equipment, on the other hand, usually does not generate significant heat or does not require infrared monitoring, so performing complex detection on secondary equipment is both unnecessary and wasteful of resources. Therefore, by using conditional inference paths based on equipment type, the average computational load of the system can be significantly reduced while ensuring the accuracy of core equipment identification.

[0070] Furthermore, step S1 also includes: The infrared image is subjected to dimensionality transformation processing to obtain three-dimensional single-channel data; The three-dimensional single-channel data is input into a preset classification model to obtain the recognition result.

[0071] It should be noted that the preset classification model is a lightweight convolutional neural network model pre-trained and deployed in the power dispatching platform, used to perform fast semantic discrimination on the input three-dimensional single-channel data. The recognition result is the semantic judgment conclusion output by the classification model, usually a probability distribution vector or discrete category label; the recognition result includes at least: whether there is power equipment in the image; if so, whether it belongs to primary equipment.

[0072] In the specific implementation, the received one-dimensional temperature data undergoes dimensionality transformation, converting it into a 640x480x1 three-dimensional array. This 640x480x1 array is then normalized and its dimensions are expanded to transform the temperature data structure into a 1x640x480x1 temperature matrix. The preprocessed temperature matrix is ​​input into a classification model for feature extraction, and the extracted neural network features are fed into a classifier to determine whether the device type in the infrared thermal image is a primary or secondary device. Temperature matrix data classified as primary device is then input into a fusion inference model for further processing.

[0073] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the method for identifying electrical equipment in infrared images. Any simple modifications based on this technical concept are within the protection scope of this application.

[0074] This application also provides a device for identifying electrical equipment in infrared images; please refer to [reference needed]. Figure 4 The device for identifying electrical equipment in infrared images includes: Acquisition module 10 is used to acquire infrared images of the power grid. The inference module 20 is used to input the infrared image into a preset fusion inference model to obtain the equipment information of all power equipment in the infrared image. The fusion inference model includes a RetinaNet sub-model that extracts temperature features after balancing positive and negative samples in the infrared image, an EfficientDet sub-model that extracts the temperature features of the main power equipment in the power equipment, and a Scaled-YOLOv4 sub-model that extracts the temperature features of local heating elements in the power equipment. The filtering module 30 is used to filter target information of target devices located at the center of the infrared image from the device information.

[0075] Optionally, the inference module 20 is further configured to input the infrared image into a preset fusion inference model, preprocess the infrared image to obtain a temperature matrix; perform positive and negative sample balancing on the temperature matrix based on the RetinaNet sub-model, and extract the first temperature feature of all power equipment in the temperature matrix; extract the second temperature feature of the main power equipment in the temperature matrix based on the EfficientDet sub-model; extract the third temperature feature of the local heating elements in the temperature matrix based on the Scaled-YOLOv4 sub-model; and perform splicing and classification processing on the first temperature feature, the second temperature feature, and the third temperature feature to obtain equipment information of all power equipment in the infrared image, wherein the equipment information includes equipment type, equipment location, and classification confidence.

[0076] Optionally, the inference module 20 is further configured to: acquire historical infrared images of the power grid; use the historical infrared images as sample data to train an initial fusion model to obtain a pre-trained fusion model, and measure the test accuracy of each sub-model in the pre-trained fusion model; establish new fully connected layers in each sub-model; perform weighted processing on the new fully connected layers based on the test accuracy to obtain an initialized fully connected layer; freeze the parameters of the pre-trained convolutional layers in the pre-trained fusion model, and perform feature extraction training on the initialized fully connected layer to obtain a fusion inference model.

[0077] Optionally, the inference module 20 is further configured to perform dimensional transformation on the infrared image based on the fusion inference model to obtain three-dimensional temperature data; perform data stacking processing on the three-dimensional temperature data to obtain three-dimensional multi-channel data; perform normalization processing on the three-dimensional multi-channel data; and perform dimensional expansion processing on the normalized three-dimensional multi-channel data to obtain a temperature matrix.

[0078] Optionally, the inference module 20 is further configured to perform weighted processing on the first temperature feature, the second temperature feature, and the third temperature feature; perform size-uniform stitching on the weighted first temperature feature, the second temperature feature, and the third temperature feature to obtain stitched features; and based on the stitched features, classify all power equipment in the infrared image to determine the equipment type, equipment location, and classification confidence level of all power equipment.

[0079] Optionally, if the identification of the power equipment in the power grid is a re-identification, the reasoning module 20 is further configured to perform Euclidean distance calculation between the temperature matrix and each historical matrix stored in the preset feature library to obtain the Euclidean distance between the temperature matrix and each of the historical matrices; based on the Euclidean distance, determine the similarity between the power equipment and the recording device corresponding to each of the historical matrices; and select the device with the highest similarity among the recording devices as the target device at the center of the infrared image.

[0080] Optionally, the acquisition module 10 is further configured to perform preliminary device identification on the infrared image and obtain identification results; if the identification result indicates that the power equipment is a primary device, then the step of inputting the infrared image into a preset fusion inference model is executed to obtain the device information of all power equipment in the infrared image.

[0081] Optionally, the acquisition module 10 is further configured to perform dimensional transformation processing on the infrared image to obtain three-dimensional single-channel data; and input the three-dimensional single-channel data into a preset classification model to obtain recognition results.

[0082] The device for identifying power equipment in infrared images provided in this application employs the method for identifying power equipment in infrared images described in the above embodiments, and can solve the technical problem of low accuracy in identifying power equipment from infrared images. Compared with the prior art, the beneficial effects of the device for identifying power equipment in infrared images provided in this application are the same as those of the method for identifying power equipment in infrared images provided in the above embodiments, and other technical features of the device for identifying power equipment in infrared images are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0083] This application provides a device for identifying electrical equipment in infrared images. The device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for identifying electrical equipment in infrared images as described in Embodiment 1 above.

[0084] The following is for reference. Figure 5 This document illustrates a structural schematic diagram of a device suitable for identifying electrical equipment in infrared images, as described in the embodiments of this application. The device for identifying electrical equipment in infrared images in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The illustrated device for identifying electrical equipment in infrared images is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0085] like Figure 5As shown, the device for identifying electrical equipment in infrared images may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for operating the device for identifying electrical equipment in infrared images. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the identification device for electrical equipment in infrared images to exchange data with other devices wirelessly or via wired communication. Although various systems for identifying electrical equipment in infrared images are shown in the figures, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0086] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0087] The device for identifying power equipment in infrared images provided in this application, employing the method for identifying power equipment in infrared images described in the above embodiments, can solve the technical problem of low accuracy in identifying power equipment from infrared images. Compared with the prior art, the beneficial effects of the device for identifying power equipment in infrared images provided in this application are the same as those of the method for identifying power equipment in infrared images provided in the above embodiments, and other technical features of this device for identifying power equipment in infrared images are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0088] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0089] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0090] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to perform the method for identifying electrical equipment in infrared images as described in the above embodiments.

[0091] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0092] The aforementioned computer-readable storage medium may be included in a device for identifying electrical equipment in infrared images; or it may exist independently and not be assembled into a device for identifying electrical equipment in infrared images.

[0093] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the identification device for electrical equipment in an infrared image, cause the identification device for electrical equipment in an infrared image to implement the aforementioned method for identifying electrical equipment in an infrared image.

[0094] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0096] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0097] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described method for identifying electrical equipment in infrared images, thereby solving the technical problem of low accuracy in identifying electrical equipment from infrared images. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the method for identifying electrical equipment in infrared images provided in the above embodiments, and will not be repeated here.

[0098] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method for identifying electrical equipment in an infrared image as described above.

[0099] The computer program product provided in this application can solve the technical problem of low accuracy in identifying power equipment from infrared images. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the method for identifying power equipment in infrared images provided in the above embodiments, and will not be repeated here.

[0100] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A method for identifying electrical equipment in infrared images, characterized in that, The method includes: Acquire infrared images of the power grid; The infrared image is input into a preset fusion inference model to obtain the equipment information of all power equipment in the infrared image. The fusion inference model includes a RetinaNet sub-model that extracts temperature features after balancing positive and negative samples in the infrared image, an EfficientDet sub-model that extracts the temperature features of the main power equipment in the power equipment, and a Scaled-YOLOv4 sub-model that extracts the temperature features of the local heating elements in the power equipment. Target information of the target device located at the center of the infrared image is filtered from the device information.

2. The method as described in claim 1, characterized in that, The step of inputting the infrared image into a preset fusion inference model to obtain equipment information of all power devices in the infrared image includes: The infrared image is input into a preset fusion inference model, and the infrared image is preprocessed to obtain a temperature matrix; After performing positive and negative sample balancing on the temperature matrix based on the RetinaNet sub-model, the first temperature feature of all power equipment in the temperature matrix is ​​extracted. Based on the EfficientDet sub-model, the temperature features of the main power equipment in the temperature matrix are extracted to obtain the second temperature features; Based on the Scaled-YOLOv4 sub-model, the temperature features of local heating elements in the temperature matrix are extracted to obtain the third temperature feature. The first temperature feature, the second temperature feature, and the third temperature feature are spliced ​​and classified to obtain equipment information of all power equipment in the infrared image of the device.

3. The method as described in claim 2, characterized in that, Before the step of inputting the infrared image into a preset fusion inference model and preprocessing the infrared image to obtain a temperature matrix, the method further includes: Obtain historical infrared images of the power grid; The historical infrared images are used as sample data to train an initial fusion model, and the test accuracy of the initial fusion model and each sub-model in the initial fusion model is obtained. Establish new fully connected layers in each of the sub-models; Based on the accuracy of each test, the new fully connected layer is weighted to obtain the initialized fully connected layer; Freeze the parameters of the pre-trained convolutional layers in the initial fusion model, and then perform feature extraction training on the initial fully connected layers to obtain the fusion inference model.

4. The method as described in claim 2, characterized in that, Before the step of extracting the first temperature features of all power devices in the temperature matrix after performing positive and negative sample balancing processing on the temperature matrix based on the RetinaNet sub-model, the method further includes: Based on the fusion inference model, the infrared image is dimensionally transformed to obtain three-dimensional temperature data; The three-dimensional temperature data is stacked to obtain three-dimensional multi-channel data; The three-dimensional multi-channel data is normalized. The normalized three-dimensional multi-channel data is subjected to dimensional expansion processing to obtain the temperature matrix.

5. The method as described in claim 2, characterized in that, The equipment information includes equipment type, equipment location, and classification confidence level. The step of performing stitching and classification processing on the first temperature feature, the second temperature feature, and the third temperature feature to obtain equipment information for all power equipment in the infrared image includes: The first temperature feature, the second temperature feature, and the third temperature feature are weighted; The first, second, and third temperature features after weighting are uniformly spliced ​​together to obtain the spliced ​​features. Based on the stitching features, all electrical devices in the infrared image are classified to determine the device type, location, and classification confidence level of each device.

6. The method as described in claim 2, characterized in that, If the identification of power equipment in the power grid is a re-identification, after the step of inputting the infrared image into a preset fusion inference model and preprocessing the infrared image to obtain a temperature matrix, the method further includes: The temperature matrix is ​​compared with each of the historical matrices stored in the preset feature library using Euclidean distance calculation to obtain the Euclidean distance between the temperature matrix and each of the historical matrices. Based on the Euclidean distance, the similarity between the power equipment and the recording equipment corresponding to each of the historical matrices is determined; The device with the highest similarity among the various recording devices is selected as the target device at the center of the infrared image.

7. The method as described in claim 1, characterized in that, Following the step of acquiring the infrared image of the power grid, the method further includes: The infrared image is subjected to preliminary device identification to obtain the identification result; If the identification result indicates that the power equipment is a primary device, then the step of inputting the infrared image into a preset fusion inference model is executed to obtain the equipment information of all power equipment in the infrared image.

8. The method as described in claim 7, characterized in that, The step of performing preliminary device identification on the infrared image to obtain the identification result includes: The infrared image is subjected to dimensionality transformation processing to obtain three-dimensional single-channel data; The three-dimensional single-channel data is input into a preset classification model to obtain the recognition result.

9. A device for identifying electrical equipment in infrared images, characterized in that, The device includes: The acquisition module is used to acquire infrared images of the power grid. The inference module is used to input the infrared image into a preset fusion inference model to obtain the equipment information of all power equipment in the infrared image. The fusion inference model includes a RetinaNet sub-model that extracts temperature features after balancing positive and negative samples in the infrared image, an EfficientDet sub-model that extracts the temperature features of the main power equipment in the power equipment, and a Scaled-YOLOv4 sub-model that extracts the temperature features of the local heating elements in the power equipment. The filtering module is used to filter target information of target devices located at the center of the infrared image from the device information.

10. A device for identifying electrical equipment in infrared images, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for identifying electrical equipment in an infrared image as described in any one of claims 1 to 8.

Images