A real-time detection method for abnormal infrared thermal imaging of substation equipment
Through lightweight deep learning models and multi-scale detection algorithms, combined with infrared image enhancement technology, the complexity and real-time problems of existing infrared image abnormality detection technology are solved, real-time accurate detection and fault evaluation of power equipment are realized, and the Test efficiency and accuracy.
Patent Information
- Application Number
- CN202011058516.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-09-30
AI Technical Summary
The existing infrared image abnormality detection technology has complex algorithms, large model calculation volume, large parameters, low accuracy and poor generalization capabilities, making it difficult to realize real-time detection of small devices, and infrared image preprocessing requires manual parameters adjustment, making it difficult to optimize.
Using a lightweight deep learning model, combined with EfficientNet and YOLOv3, a real-time deep learning detection model for thermal failure of mobile power equipment is constructed, and image enhancement is performed through NSCT transformation, adaptive histogram equalization and adaptive genetic algorithm to optimize the channel of the detection model. and parameters to achieve end-to-end real-time classification positioning and assessment of fever severity.
Real-time detection on front-end artificial intelligence embedded devices is realized, detection accuracy and efficiency are improved, model parameters and calculations are reduced, and heating positions are accurately detected in real time and provided fault level warnings.
Smart Images

Figure CN112233073B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of abnormality detection of power equipment, and in particular to a real-time detection method for abnormalities of substation equipment using infrared thermal imaging. Background Art
[0002] Substations are important hubs in the power grid responsible for converting voltage and distributing electrical energy. Their safe and stable operation is crucial and directly related to the safety of the power system and the stability of power supply. Long-term operation of power equipment or environmental factors can cause problems such as poor contact and overload in key parts, causing heating in some areas of the equipment. According to relevant statistics, more than half of faulty power equipment will have abnormal heating.
[0003] Infrared thermal imaging of power equipment detects the infrared radiation energy emitted by power equipment, converts the thermal signal into an electrical signal, and then obtains the thermal image of the power equipment after processing the electrical signal. Power infrared detection has the characteristics of non-stop power supply, non-contact, and mature technology. It can detect potential hidden dangers and faults of power equipment and has been widely used in the field of thermal anomaly detection of power equipment.
[0004] In the existing technology, most of the research on anomaly detection in infrared images is based on traditional machine learning algorithms, which have problems such as complex algorithms, large model calculation amount and parameter amount, low accuracy, poor generalization ability, rigid model, and difficulty in conducting real-time anomaly detection on small devices.
[0005] In addition, in anomaly detection, the preprocessing of infrared images directly affects the detection accuracy. The current mainstream methods for infrared image processing mainly use wavelet transform, Contourlet transform, etc. to decompose the image, and combine certain algorithms (such as general histogram equalization algorithm, filtering algorithm) to process the high-frequency sub-band and the low-frequency sub-band and then reconstruct the image. Although the image processing effect is improved, these algorithms still have problems such as the need for manual parameter adjustment and difficulty in optimization. Summary of the invention
[0006] The purpose of this application is to use a lightweight deep learning model, embed this network model into the host computer system, realize real-time detection and analysis of infrared image anomalies of power equipment, and achieve truly intelligent inspection.
[0007] Combined with the characteristics of infrared images of substation equipment and equipment and heating areas, an effective infrared image enhancement method is used to preprocess the data set to improve the accuracy of anomaly detection. The basic idea of the single-stage real-time detection model YOLOv3 is adopted, and its backbone extraction network is replaced with EffcientNet, so as to build a lightweight mobile terminal power equipment thermal fault real-time deep learning detection model. Deployed on the mobile terminal, it can quickly and accurately obtain the location of abnormal heating points of power equipment and its heating severity, realizing intelligent real-time inspection of thermal faults of power equipment.
[0008] The technical solution of the present application is to provide a real-time detection method for infrared thermal imaging anomalies of substation equipment, the method comprising: step 1, according to the infrared image collected in the sample area, target frame annotation and image enhancement are performed to generate a sample data set, wherein the target frame includes the category and location of the power equipment, specifically comprising: step 21, inputting the infrared image in the training data set into a 3×3 convolution layer in the EfficientNet network to adjust the input dimension; step 22, performing a convolution operation with a step of 2 on the infrared image after dimension adjustment, and performing a moving flip bottleneck convolution according to a preset stage to extract a feature map of a specified channel in the EfficientNet network; step 23, compressing and Adaptive adjustment, tensor splicing the compressed and adjusted YOLOv3 model with the feature map of the specified channel to generate the initial detection model; step 2, adjust the number of channels between the neural network model and the single-stage real-time detection model to generate the initial detection model, wherein the model parameters of the neural network model are determined by preset constraints; step 3, train and optimize the initial detection model according to the sample data set, and record the optimized initial detection model as the infrared thermal imaging anomaly detection model; step 4, obtain the infrared image of the area to be detected, use the infrared thermal imaging anomaly detection model to perform anomaly detection on the infrared image of the area to be detected, and output the predicted box of the abnormal area in the infrared image of the area to be detected and the maximum temperature value. The predicted box includes position information, and the position information includes the prediction box coordinates as well as the height and width. The position information is determined by the prediction loss function L CIoU Calculate the prediction loss function L CIoU The calculation formula is:
[0009]
[0010] Where ρ is the Euclidean distance, c is the diagonal distance between the minimum bounding rectangle of the prediction box and the target box, and b, b gt are the center points of the prediction box and the target box respectively, w, w gt are the widths of the prediction box and the target box, h and h respectively. gtare the heights of the prediction box and the target box, α is a balance ratio parameter, and v is a ratio consistency parameter; the prediction box includes fault type information, which is determined by the prediction box corresponding to the maximum classification confidence in the current prediction box. The calculation formula of the classification confidence is:
[0011]
[0012] In the formula, m, b I They are the prediction box M with the highest confidence and the current prediction box B I The center point, S I is the classification confidence, ∈ is the preset threshold, M is the prediction box with the highest confidence, B I The current prediction box.
[0013] In any of the above technical solutions, further, a prediction module is provided in the initial detection model, and the prediction module is used to output the prediction frame according to the prior frame, and the adjustment process of the prior frame includes: step 201, according to the preset K value, randomly select K points in the sample data set as cluster centers, and calculate the clustering error, and record the K value corresponding to the maximum point of the second-order derivative of the clustering error as the initial cluster number; step 202, according to the initial cluster number, determine the corresponding cluster center point, and calculate the distance between the target frame of each infrared image in the sample data set and the cluster center point in turn; step 203, assign the target frame to the cluster center point corresponding to the minimum distance, and perform splitting and merging processing, update the cluster center, and adjust the cluster number; step 204, use iterative operation, re-execute steps 202 and 203, and when it is determined that the adjusted cluster number converges, determine the prior frame according to the converged cluster number.
[0014] In any of the above technical solutions, further, the calculation formula of the clustering error SSE is:
[0015]
[0016] In the formula, C I is the i-th cluster, p is the cluster C I The sample points in M I It is cluster C I The center of mass.
[0017] In any of the above technical solutions, further, in step 4, the method for calculating the maximum temperature value includes: step 41, extracting the highest temperature and the lowest temperature in the infrared image of the area to be detected, and determining the RGB brightness value of each pixel in the infrared image of the area to be detected; step 42, converting the infrared image of the area to be detected into a grayscale image according to the RGB brightness value of each pixel, and respectively determining the first grayscale value and the second grayscale value corresponding to the highest temperature and the lowest temperature; step 43, calculating the temperature calculation value of each pixel in the abnormal area in the grayscale image according to the highest temperature, the lowest temperature, the first grayscale value, and the second grayscale value, sorting the temperature calculation values, and selecting the maximum value of the temperature calculation values as the highest temperature value of the abnormal area in the infrared image of the area to be detected.
[0018] In any of the above technical solutions, further, in step 1, image enhancement is performed on the collected infrared image, specifically including: performing NSCT transformation on the infrared image to obtain multi-scale and multi-directional high-frequency sub-band coefficients and low-frequency sub-band coefficients; performing contrast enhancement processing on the low-frequency sub-band coefficients through an adaptive histogram equalization method; performing fuzzy enhancement on the high-frequency sub-band to enhance the contrast between the target and the background, wherein the fuzzy parameters of the fuzzy enhancement are determined by an adaptive genetic algorithm; and using an inverse NSCT transform to reconstruct the enhanced high-frequency sub-band coefficients and low-frequency sub-band coefficients to achieve image enhancement.
[0019] In any of the above technical solutions, further, the method also includes: generating a warning level according to the maximum temperature value of the abnormal area in the infrared image of the area to be detected and a preset level threshold, and performing a fault level warning of the power equipment.
[0020] The beneficial effects of this application are:
[0021] The technical solution in this application adopts the lightweight neural network model Efficientnet to perform end-to-end real-time classification positioning and fever severity assessment. This method meets the practical requirements in terms of detection accuracy and reasoning speed, and realizes its application in front-end artificial intelligence embedded devices.
[0022] This application uses the lightweight neural network model Efficientnet as the backbone network, which not only greatly reduces the number of model parameters and calculations, but also achieves the purpose of improving indicators by comprehensively optimizing network width, deepening the network and resolution. Its recognition accuracy exceeds that of other convolutional network models. It can also be deployed on the front end to complete detection tasks in real time, improving the efficiency of anomaly detection.
[0023] This application also uses the multi-scale detection algorithm YOLOv3 according to the type of power equipment and the characteristics of the heating point area, which can accurately detect the heating locations of different sizes in real time.
[0024] In a preferred implementation of the present application, the severity of the heating point can be analyzed and evaluated, so that the inspection personnel can timely understand the severity of the heating and troubleshoot the fault in time. In particular, when the temperature of the detected abnormal area does not exceed the set threshold, it is reminded that manual inspection is required to accurately check whether there is a fault according to the reference standard to eliminate the hidden danger. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The advantages of the above and / or additional aspects of the present application will become apparent and easily understood in the description of the embodiments in conjunction with the following drawings, in which:
[0026] Figure 1 is a schematic flow chart of a method for real-time detection of abnormalities of substation equipment by infrared thermal imaging according to an embodiment of the present application;
[0027] Figure 2 is a schematic diagram of an infrared image of an electric power device according to an embodiment of the present application;
[0028] Figure 3 is a schematic diagram of an infrared image enhancement process according to an embodiment of the present application;
[0029] Figure 4 is a schematic diagram of an Efficientnet network structure according to an embodiment of the present application;
[0030] Figure 5 is a schematic diagram of a partial framework of an infrared thermal imaging anomaly detection model according to an embodiment of the present application;
[0031] Figure 6 It is a schematic diagram of an infrared image anomaly detection process according to an embodiment of the present application. DETAILED DESCRIPTION
[0032] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0033] In the following description, many specific details are elaborated to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present application is not limited to the specific embodiments disclosed below.
[0034] At present, infrared image enhancement processing mainly uses wavelet transform, Contourlet transform, etc. to decompose the image, and combines certain algorithms to realize the separate processing of high-frequency sub-bands and low-frequency sub-bands. In particular, the use of fuzzy enhancement improves the effect. However, the fuzzy set parameters need to be manually randomly input and are not optimized. This embodiment uses multi-scale and multi-directional NSCT (Nonsubsampled Contourlet) transform to decompose the image, and according to the difference in image retention information, combines adaptive histogram equalization and adaptive genetic algorithm optimization of fuzzy enhancement method to process high-frequency images and low-frequency images respectively, solves the problem that fuzzy set parameters need to be manually randomly input and are not optimized, and has strong applicability, can perform image enhancement processing on infrared images of different power equipment, and achieve better results.
[0035] like Figure 1 As shown, this embodiment provides a method for real-time detection of abnormalities of substation equipment by infrared thermal imaging, the method comprising:
[0036] Step 1: perform target frame annotation and image enhancement based on the infrared image collected in the sample area to generate a sample data set, wherein the target frame includes the type and location of the power equipment;
[0037] In this embodiment, a plurality of infrared images of power equipment, such as 2100 images, are collected by using an infrared thermal imager carried by a drone, an inspection robot, etc., wherein the types of power equipment include: circuit breakers, disconnectors, transformers, bushings, current transformers, lightning arresters, etc., and the corresponding infrared images are as follows: Figure 2 Then, the collected infrared images are sampled and normal images are distinguished from abnormal images, and the number of abnormal images is set to 680.
[0038] like Figure 3 As shown, this embodiment uses non-subsampled contourlet transform (NSCT), contrast-limited adaptive histogram equalization (CLHE), adaptive genetic algorithm, and fuzzy algorithm to sequentially enhance the collected infrared image.
[0039] Firstly, the infrared image is transformed by NSCT to obtain multi-scale and multi-directional high-frequency sub-band coefficients and low-frequency sub-band coefficients, which is conducive to better maintaining the edge information and contour structure of the image and enhancing the translation invariance of the image.
[0040] Secondly, since the low-frequency sub-band coefficients contain a large amount of image information, the low-frequency sub-band coefficients are contrast enhanced by an adaptive histogram equalization method to avoid excessive amplification of noise.
[0041] Again, since the high-frequency subband coefficients contain the details and edge information of the image, the high-frequency subband is fuzzy enhanced. The fuzzy enhancement algorithm can effectively stretch the grayscale range of the image, enhance the contrast between the target and the background, decompose the infrared target area and the background area, and, in the fuzzy enhancement process, optimize the fuzzy enhancement parameters through an adaptive genetic algorithm.
[0042] Finally, the NSCT inverse transform is used to reconstruct the image and obtain the enhanced image of the infrared image, thus achieving the enhancement of the infrared image.
[0043] The enhanced infrared images are processed by cropping, proportional scaling or zero-filling, and the size of the infrared images is unified to 416×416px. The images are input into conventional image processing software for target frame annotation, that is, different labels are marked on the abnormal heating area and the type and location of the power equipment in the image to form a target frame corresponding to the infrared image.
[0044] The infrared images processed as above are used as sample data sets to train and optimize the detection model constructed subsequently.
[0045] In this embodiment, the sample data set is set to include three types of data sets: training, verification, and test, and the ratio between the three is 3:1:1.
[0046] Step 2, adjusting the number of channels between the neural network model and the single-stage real-time detection model to generate an initial detection model, wherein the model parameters of the neural network model are determined by preset constraints;
[0047] In this embodiment, the YOLOv3 model is used as the single-stage real-time detection model, and the ultra-lightweight EfficientNet is used as the neural network model.
[0048] DarkNet-53 is used as the backbone feature extraction network in the original YOLOv3 algorithm. DarkNet-53 reduces network costs while improving detection speed and solves the gradient problem of deep networks. However, the disadvantage is that the number of network layers is too deep, resulting in too many network parameters and too much calculation.
[0049] In order to achieve real-time anomaly detection on the mobile terminal and make the network lightweight, the ultra-lightweight EfficientNet network is introduced in this embodiment. The network uses a composite expansion method to simultaneously expand the three dimensions of the convolutional network, depth, width, and resolution. Unlike traditional methods, the dimensions of the network are not arbitrarily scaled, which effectively improves the training results of the existing network on large-scale computing resources, with fewer network parameters, faster inference speed, and higher accuracy.
[0050] In order to optimize the model parameters in the EfficientNet network, the following preset constraints are set in this embodiment:
[0051]
[0052] stα·β 2 γ 2 ≈2
[0053] α≥1,β≥1,γ≥1
[0054] In the formula, α, β, and γ are model parameters in the EfficientNet network, which measure the proportion of depth, width, and resolution respectively. To adjust the parameters.
[0055] When calculating the model parameters α, β, and γ, first set the adjustment parameters Through grid search, the optimal model parameters α, β, and γ that meet the preset constraints are obtained as the basic model EfficientNet-B0 of the EfficientNet network.
[0056] After that, the optimal model parameters α, β, γ are fixed and different EfficientNet-B1,...,EfficientNet-B7 are obtained, that is, the basic model EfficientNet-B0 is scaled under different number of operations and running memory constraints, where α is multiplied by the number of module repetitions in each stage of EfficientNet-B0 to obtain a deeper network architecture; β is multiplied by the number of cores input to each convolution operation in EfficientNet-B0 to obtain a more expressive network model; γ controls the length and width of the network's input image.
[0057] You can choose a network with appropriate size and number of parameters based on your needs to obtain better processing results and improve the applicability of the model to different scenarios.
[0058] In this embodiment, in order to pursue a lightweight network model for application in embedded devices, EfficientNet-B0, EfficientNet-B1, and EfficientNet-B2 models may be used, with the EfficientNet-B0 model being preferred.
[0059] like Figure 4As shown in the figure, due to the large difference in the number of network channels between the EfficientNet network and the YOLOv3 model, the output of the EfficientNet network cannot be directly introduced into the YOLOv3 model. Therefore, the number of channels between the two needs to be adjusted.
[0060] This embodiment shows a method for generating an initial detection model based on an EfficientNet network and a YOLOv3 model, and the method specifically includes:
[0061] Step 21, input the infrared image in the training data set into a 3×3 convolutional layer in the EfficientNet network to adjust the input dimension, wherein the training data set is composed of a randomly selected part (3 / 5) of the sample data set.
[0062] In step 22, a convolution operation with a stride of 2 is performed on the dimensionally adjusted infrared image, and a moving flip bottleneck convolution is performed according to a preset stage to extract a feature map of a specified channel in the EfficientNet network.
[0063] In this embodiment, in order to reduce information loss, during the feature map extraction process, an infrared image with a size of 416×416px is input into the EfficientNet backbone network, and is first converted into the input dimension required by the mobile flip bottleneck convolution MBConv through a conv3×3 convolution layer.
[0064] Moreover, in the initial downsampling process, the conventional maximum pooling is not used. Instead, a convolution operation with a stride of 2 is performed on the infrared image, and then the moving flip bottleneck convolution (MBConv) is performed 1, 2, 2, 3, and 3 times according to the preset stages to extract feature maps of different scales, which are denoted as P1, P2, P3, P4, and P5 respectively.
[0065] Through a large amount of data calculation and verification, feature maps of different combinations and arrangements were tested. During the test, it was found that when P3, P4, and P5 with sizes of 52, 26, and 13 were selected as the input of the next stage, the obtained infrared thermal imaging anomaly detection model had the best effect on real-time detection of infrared image anomalies of power equipment.
[0066] Therefore, the feature maps P3, P4, and P5 in the EfficientNet network are set as specified inputs and input into the YOLOv3 model.
[0067] Step 23, compress and adaptively adjust the number of channels of the YOLOv3 model, perform tensor splicing contact between the compressed and adjusted YOLOv3 model and the extracted feature map of the specified channel to generate an initial detection model.
[0068] Specifically, Figure 5 As shown in the figure, unlike the general network structure, when performing tensor splicing contact, considering that the output of the backbone network EfficientNet network is quite different from the number of channels of the YOLOV3 network, the number of channels of the two are 40, 112, 320 and 256, 512, 1024 respectively, overfitting is prone to occur. Therefore, the original number of channels C of the YOLOV3 network is compressed to C / 4, and the left and right thresholds are set based on this, and the number of channels is adaptively adjusted in this interval to achieve the connection between the feature map of the specified channel in the EfficientNet network and the YOLOv3 model.
[0069] In the process of tensor splicing contact, the feature map P5 is first passed through the YOLOblock module (5 times of 1x1 convolution), 3x3 convolution layer conv, and 1x1 convolution layer conv in sequence to obtain the feature map Feature map1.
[0070] The feature map P5 after passing through the YOLOblock module is upsampled, and then tensor spliced with the feature map P4 to obtain the feature map P6. The feature map P6 is sequentially passed through the YOLOblock module, the 3x3 convolution layer conv, and the 1x1 convolution layer conv to obtain the feature map Feature map2.
[0071] Then, the feature map P6 after passing through the YOLOblock module is upsampled and concatenated with the feature map P3. It passes through the YOLOblock module, the 3x3 convolution layer conv, and the 1x1 convolution layer conv in sequence to obtain the feature map Feature map3.
[0072] By taking feature map 1, feature map 2, and feature map 3 as input and inputting them into the prediction module, the initial detection model can be constructed.
[0073] It should be noted that the priori box required by the prediction module for abnormal prediction can be implemented using the priori box in the existing YOLOV3 network. The existing YOLOV3 network uses the K-means clustering algorithm to obtain the size of the priori box, that is, the most likely object shape obtained by clustering statistics from the labeled data, and finally clusters 9 priori boxes, setting 3 prior boxes for each downsampling scale.
[0074] In the process of determining the prior box, the K-means clustering algorithm has two problems: first, K points in the data set are randomly selected as cluster centers at the beginning, and the value of K needs to be determined manually in advance; second, once the value of the cluster number K is determined, it cannot be changed during the entire algorithm process.
[0075] In this embodiment, in order to improve the accuracy of the prediction module in predicting the abnormality of infrared thermal imaging, the setting of the prior frame in the prediction module is adjusted. The specific process is as follows:
[0076] Step 201, according to the preset K value, randomly select K points in the sample data set as cluster centers, and calculate the clustering error, and record the K value corresponding to the maximum value of the second-order derivative of the clustering error as the initial cluster number, wherein the calculation formula of the clustering error is:
[0077]
[0078] In the formula, C I is the i-th cluster, p is the cluster C I The sample points in the target box, M I It is cluster C I The centroid of cluster C I SSE is the mean of all samples in the clustering error, which represents the quality of clustering effect.
[0079] Specifically, the clustering error SSE is the sum of squared distance errors from points in the cluster to the center point for each K value. Data testing found that as the value of the cluster number K increases, sample classification will be more accurate, the degree of aggregation of each cluster will gradually increase, and the clustering error SSE will decrease accordingly. In addition, when the K value is less than the actual number of clusters, the increase in the K value will greatly increase the degree of aggregation of the clusters, so the decrease in the clustering error SSE will be large. When the K value reaches the actual number of clusters, the degree of aggregation obtained by increasing the K value will rapidly decrease, so the decrease in the clustering error SSE becomes very gentle. At this time, the K value corresponding to the point with the largest second-order derivative of the clustering error SSE is selected and recorded as the initial number of clusters.
[0080] Step 202, according to the initial cluster number, determine the corresponding cluster center point, and calculate the distance between the target box of each infrared image (cluster sample) in the sample data set and the cluster center point in turn;
[0081] Step 203, assign the target frame to the cluster center point corresponding to the minimum distance value, and perform splitting and merging processing, update the cluster center, and adjust the number of clusters.
[0082] In this embodiment, the iterative self-organizing data analysis method (ISODATA) algorithm is used to replace the K-means algorithm on the premise of obtaining the initial cluster number. The ISODATA algorithm can adjust the cluster number K by performing two processing operations, splitting and merging, according to the actual situation of each category during the training process.
[0083] The merging process is to merge the two categories into one category when the number of samples in one category of the clustering result is too small or the distance between two categories is too close. The splitting process is to split the category into two categories when the intra-category variance of one category in the clustering result is too large.
[0084] Step 204, using iterative operation, re-execute step 202 and step 203, when it is determined that the adjusted number of clusters converges, determine the prior frame according to the converged number of clusters.
[0085] Specifically, according to the characteristics of power equipment and heating point areas, the sizes of the prior boxes finally clustered are sorted and assigned to three feature maps of different scales.
[0086] Distribution rule: large-scale feature maps use smaller prior boxes to detect small objects, and small-scale feature maps use smaller prior boxes to detect large objects. When the final cluster number K is a multiple of 3, the prior boxes are evenly distributed. When K is not a multiple of 3, the 13×13 and 26×26 scales are obtained respectively. The rest are assigned to feature maps of size 52x52.
[0087] Step 3: train and optimize the initial detection model according to the sample data set, and record the optimized initial detection model as the infrared thermal imaging anomaly detection model.
[0088] In this embodiment, after the initial detection model is generated, the training data set in the sample data set is first input as input to the initial detection model, and the initial detection model is trained. In order to achieve faster convergence and better performance, the Adam optimizer is used in the training process, and the dropout method is used to prevent the model from overfitting.
[0089] Afterwards, the verification and test data sets in the sample data set are input into the trained initial detection model in turn to optimize it. The optimized model is recorded as the infrared thermal imaging anomaly detection model, and the prediction box corresponding to the infrared image of the power equipment is output to realize real-time detection of anomalies in the infrared image of the power equipment.
[0090] Step 4: Obtain an infrared image of the area to be detected, use the infrared thermal imaging anomaly detection model to perform anomaly detection on the infrared image of the area to be detected, and output the predicted box of the abnormal area in the infrared image of the area to be detected and the maximum temperature value.
[0091] In this embodiment, the prediction box includes position information and fault type information, wherein the position information includes the prediction box coordinates as well as the height and width, and the fault type information is used to determine the fault type of the area to be detected.
[0092] It should be noted that for the prediction of the same image, many prediction boxes will be obtained, each of which includes location information and fault type information. By screening the prediction boxes, an optimal prediction box is retained as the output prediction box.
[0093] When determining the position information of the prediction box, the intersection over union (IoU) is usually used in the prior art to determine the degree of overlap between two boxes. The prediction box prediction function in the original algorithm Yolo v3 adopts IoU loss, that is, the intersection over union (IoU) is used as the loss function for regressing the prediction box. This method cannot distinguish different alignment methods between two objects. Moreover, if the two targets do not overlap, the intersection over union (IoU) will be 0. At this time, no matter how far or near the distance between the prediction box and the target box is, the intersection over union (IoU) cannot be reflected.
[0094] Moreover, if the intersection over union (IoU) is used as the loss function, the gradient of the loss function will be 0 when optimizing the loss function, that is, the loss function is not differentiable, so the loss function cannot be optimized. Therefore, the original algorithm has a great drawback.
[0095] In this embodiment, three geometric characteristics of the prediction box and the target box are considered at the same time, including: overlapping area, center point distance, and aspect ratio. The prediction box coordinates, height, and width of the prediction box are calculated using the prediction loss function. The prediction loss function L of the prediction box is CIoU The calculation formula is:
[0096]
[0097] Where ρ is the Euclidean distance, c is the diagonal distance between the minimum bounding rectangle of the prediction box and the target box, and b, b gt are the center points of the prediction box and the target box, w and w gt are the width of the prediction box and the target box, h and h respectively. gt are the heights of the predicted box and the target box, α is the balance ratio parameter, and v is the ratio consistency parameter, which is used to measure the ratio consistency between the predicted box and the target box.
[0098] When determining the fault type information of the prediction box, the classification confidence function is introduced. Since the conventional algorithm adopts the traditional NMS (non-maximum suppression) method, IoU is used as an indicator to suppress redundant detection boxes, that is, the detection boxes are sorted by score, and then the box with the highest score is retained, while other boxes with an overlapping area greater than a certain ratio with the box are deleted. However, since only the overlapping area is considered, incorrect suppression often occurs.
[0099] In this embodiment, not only the overlapping area but also the information of the center point position between the prediction box and the target box is considered, and the classification confidence function is adjusted. The adjusted classification confidence function is:
[0100]
[0101] Where ρ is the Euclidean distance, c is the diagonal distance between the minimum bounding rectangle of the prediction box and the target box, m, b I They are the prediction box M with the highest confidence and the current prediction box B I The center point, S I is the classification confidence, ∈ is the preset threshold.
[0102] That is, when DIoU(M,B I ) is less than the preset threshold ∈, retain the current classification confidence and retain the current prediction box B I ; When DIoU(M,B I ) is greater than or equal to the preset threshold ∈, the current classification confidence is set to 0 and the current prediction box B is deleted I .
[0103] Furthermore, step 4 also includes: providing an early warning of a power equipment failure based on the maximum temperature value of the abnormal area in the infrared image of the area to be detected and a preset level threshold.
[0104] Specifically, the infrared image of the area to be detected is input into the infrared thermal imaging anomaly detection model, and the model performs anomaly detection on the power equipment in the infrared image of the area to be detected. During the detection process, not only can the abnormal heating point area be detected in real time, but also the severity of the equipment heating can be analyzed in time according to the temperature data.
[0105] When the infrared thermal imaging anomaly detection model predicts an abnormal hot spot, the prediction box is used to select the infrared image of the detection area, and the maximum temperature at the abnormal hot spot is calculated through the corresponding first-order fitting function between temperature and gray value. The process specifically includes:
[0106] Step 41, extracting the highest temperature and the lowest temperature in the infrared image of the area to be detected, and determining the RGB brightness value of each pixel in the infrared image of the area to be detected;
[0107] Step 42, according to the RGB brightness value of each pixel point, the infrared image of the area to be detected is converted into a grayscale image, and the first grayscale value and the second grayscale value corresponding to the highest temperature and the lowest temperature are determined respectively, wherein the calculation formula for converting the infrared image into the grayscale image is:
[0108] Gray(i,j)=0.11*R(i,j)+0.59*G(i,j)+0.3*B(i,j)
[0109] Where (i, j) is the pixel position coordinate, Gray(i, j) is the pixel gray value at the coordinate (i, j), R(·) is the red component in the RGB brightness value of the pixel, G(·) is the green component in the RGB brightness value of the pixel, and B(·) is the blue component in the RGB brightness value of the pixel;
[0110] Step 43, according to the highest temperature, the lowest temperature, the first gray value, and the second gray value, calculate the temperature calculated value of each pixel in the abnormal area in the gray image, sort the temperature calculated values, and select the maximum value of the temperature calculated values as the highest temperature value of the abnormal area in the infrared image of the area to be detected, wherein the calculation formula of the temperature calculated value is:
[0111] T=(t1-t2) / (x1-x2)X+(t1-(t1-t2) / (x1-x2)x1)
[0112] Wherein, T is the calculated temperature value, t1 is the highest temperature, t2 is the lowest temperature, x1 is the first gray value, x2 is the second gray value, and X is the gray value of each pixel in the abnormal area of the gray image.
[0113] In order to improve the accuracy of real-time detection of infrared thermal imaging anomalies, this embodiment also sets three temperature warning levels, and the corresponding temperatures are 55°C, 80°C, and 110°C respectively. The corresponding relationship between the maximum temperature of the abnormal area and the severity level is shown in Table 1.
[0114] Table 1
[0115]
[0116] like Figure 6 As shown, when abnormal hot spots are detected in the power equipment, emergency maintenance measures of different levels are carried out according to the predicted warning level to ensure the normal operation of the power equipment.
[0117] In particular, if it is determined that there is an abnormal heating pad in the power equipment, but when its temperature does not exceed 55°C, it is necessary to manually analyze the infrared image data or conduct on-site inspections based on the type of power equipment detected, and refer to the "DL / T 64-2016 Infrared Diagnosis Application Specification for Live Equipment Detection" standard to promptly troubleshoot and eliminate hidden dangers.
[0118] The above describes in detail the technical solution of the present application in combination with the accompanying drawings. The present application proposes a real-time detection method for infrared thermal imaging anomalies of substation equipment, including: step 1, target frame annotation and image enhancement are performed according to the infrared image collected from the sample area to generate a sample data set; step 2, the number of channels between the neural network model and the single-stage real-time detection model is adjusted to generate an initial detection model; step 3, the initial detection model is trained and optimized according to the sample data set, and the optimized initial detection model is recorded as the infrared thermal imaging anomaly detection model; step 4, the infrared image of the area to be detected is obtained, and the infrared thermal imaging anomaly detection model is used to perform anomaly detection on the infrared image of the area to be detected, and the predicted box and the highest temperature value of the abnormal area in the infrared image of the area to be detected are output. Through the technical solution in the present application, a lightweight deep learning model is adopted to realize real-time detection of infrared image anomalies of power equipment and intelligent real-time inspection.
[0119] The steps in this application can be adjusted in order, combined, and deleted according to actual needs.
[0120] The units in the device of the present application can be combined, divided and deleted according to actual needs.
[0121] Although the present application is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present application. The scope of protection of the present application is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present application.
Claims
1. A method for real-time detection of abnormalities in infrared thermal imaging of substation equipment, characterized in that: The method comprises: Step 1: perform target frame annotation and image enhancement based on the infrared image collected in the sample area to generate a sample data set, wherein the target frame includes the type and location of the power equipment; Step 2, adjusting the number of channels between the neural network model and the single-stage real-time detection model to generate an initial detection model, wherein the model parameters of the neural network model are determined by preset constraints, specifically including: Step 21, input the infrared image in the training data set into a 3×3 convolutional layer in the EfficientNet network to adjust the input dimension; Step 22, performing a convolution operation with a step of 2 on the dimensionally adjusted infrared image, and performing a moving flip bottleneck convolution according to a preset stage to extract a feature map of a specified channel in the EfficientNet network; Step 23, compressing and adaptively adjusting the number of channels of the YOLOv3 model, performing tensor splicing of the compressed and adjusted YOLOv3 model and the feature map of the specified channel to generate the initial detection model; Step 3: training and optimizing the initial detection model according to the sample data set, and recording the optimized initial detection model as the infrared thermal imaging anomaly detection model; Step 4, obtaining an infrared image of the area to be detected, performing anomaly detection on the infrared image of the area to be detected using the infrared thermal imaging anomaly detection model, and outputting a predicted frame and a maximum temperature value of the abnormal area in the infrared image of the area to be detected; The prediction box includes position information, which includes the prediction box coordinates, height, and width. The position information is determined by the prediction loss function L CIoU Calculate the prediction loss function L CIoU The calculation formula is: Where ρ is the Euclidean distance, c is the diagonal distance between the minimum bounding rectangle of the prediction box and the target box, and b, b gt are the center points of the prediction box and the target box respectively, w, w gt are the widths of the prediction box and the target box, h and h respectively. gt are the heights of the predicted box and the target box, α is the balance ratio parameter, v is the ratio consistency parameter, and IoU is the intersection over union ratio; The prediction frame includes fault type information, and the fault type information is determined by the prediction frame corresponding to the maximum classification confidence in the current prediction frame. The calculation formula of the classification confidence is: In the formula, m, b I They are the prediction box M with the highest confidence and the current prediction box B I The center point, S I is the classification confidence, ∈ is the preset threshold, M is the prediction box with the highest confidence, B I The current prediction box.
2. The method for real-time detection of abnormalities of substation equipment by infrared thermal imaging according to claim 1, characterized in that: The initial detection model is provided with a prediction module, and the prediction module is used to output the prediction frame according to the prior frame, and is characterized in that the adjustment process of the prior frame includes: Step 201, randomly selecting K points in the sample data set as cluster centers according to a preset K value, and calculating the clustering error, and recording the K value corresponding to the maximum value of the second-order derivative of the clustering error as the initial cluster number; Step 202, determining the corresponding cluster center point according to the initial cluster number, and calculating the distance between the target frame of each infrared image in the sample data set and the cluster center point in turn; Step 203, assigning the target frame to the cluster center point corresponding to the minimum distance value, performing splitting and merging processing, updating the cluster center, and adjusting the number of clusters; Step 204, using iterative operation, re-execute step 202 and step 203, when it is determined that the adjusted number of clusters converges, determine the priori frame according to the converged number of clusters.
3. The method for real-time detection of abnormalities of substation equipment by infrared thermal imaging according to claim 2, characterized in that: The calculation formula of the clustering error SSE is: In the formula, C I is the i-th cluster, p is the cluster C I The sample points in M I It is cluster C I The center of mass.
4. The method for real-time detection of abnormalities of substation equipment by infrared thermal imaging according to claim 1, characterized in that: In step 4, the method for calculating the maximum temperature value includes: Step 41, extracting the highest temperature and the lowest temperature in the infrared image of the area to be detected, and determining the RGB brightness value of each pixel in the infrared image of the area to be detected; Step 42, converting the infrared image of the area to be detected into a grayscale image according to the RGB brightness values of each pixel point, and determining a first grayscale value and a second grayscale value corresponding to the highest temperature and the lowest temperature, respectively; Step 43, calculate the temperature calculation value of each pixel in the abnormal area in the grayscale image according to the maximum temperature, the minimum temperature, the first grayscale value, and the second grayscale value, sort the temperature calculation values, and select the maximum value of the temperature calculation values as the maximum temperature value of the abnormal area in the infrared image of the area to be detected.
5. The method for real-time detection of abnormalities of substation equipment by infrared thermal imaging according to any one of claims 1 to 4, characterized in that: In step 1, the collected infrared image is enhanced, specifically including: Performing NSCT transformation on the infrared image to obtain multi-scale and multi-directional high-frequency sub-band coefficients and low-frequency sub-band coefficients; Performing contrast enhancement processing on the low-frequency subband coefficients by an adaptive histogram equalization method; Performing fuzzy enhancement on the high frequency sub-band to enhance the contrast between the target and the background, wherein the fuzzy parameters of the fuzzy enhancement are determined by an adaptive genetic algorithm; The NSCT inverse transform is used to reconstruct the enhanced high-frequency sub-band coefficients and low-frequency sub-band coefficients to achieve image enhancement.
6. The method for real-time detection of abnormalities of substation equipment by infrared thermal imaging according to any one of claims 1 to 4, characterized in that: The method further includes: generating a warning level according to the maximum temperature value of the abnormal area in the infrared image of the area to be detected and a preset level threshold, and performing a fault level warning of the electric power equipment.
Citation Information
Cited By
Heat flow field anomaly detection method based on thermal imaging sequence
CN121616575A
A thermal flow field anomaly detection method based on thermal imaging sequences
CN121616575B
