Heating defect detection method and device, nonvolatile storage medium and electronic equipment

Abstract

The invention discloses a heating defect detection method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: carrying out edge feature extraction on an infrared image comprising power equipment to obtain an edge contour; performing infrared feature extraction on the infrared image to obtain a feature vector; performing dimension reduction mapping on the feature vector by adopting a hash function to obtain a target hash code; performing temperature extraction based on a first pixel point included in the edge contour to obtain a temperature feature; and determining a heating defect detection result of the power equipment based on the target hash code and the temperature characteristics. The technical problem that the heating defect detection efficiency is not ideal in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology and the field of power technology. Specifically, it relates to a method, device, non-volatile storage medium and electronic device for detecting heating defects. Background Art

[0002] With the development of the smart grid and the continuous promotion of new energy grid connection construction, the safe and stable operation of the power grid system has become the key goal of the current power grid system construction. Online monitoring, defect diagnosis, and maintenance and repair constitute the main content of the online condition detection of electrical equipment. Infrared detection technology is an important means to detect heating defects of high-voltage electrical equipment in the live state. Its purpose is to detect the heating problems of power equipment at an early stage, avoid equipment damage and breakdown due to overheating, and cause serious consequences such as power outages. It has been applied in substations of different voltage levels.

[0003] Currently, intelligent infrared analysis usually adopts infrared spectrum template calibration or infrared image deep learning methods. Infrared image template calibration uses a standard template as a reference picture to frame the target detection object, and parses the corresponding temperature matrix file of the detected infrared spectrum to obtain the highest temperature for alarm judgment. In related technologies, detection errors are likely to occur when there is a deviation between the collected spectrum and the reference picture. The infrared image deep learning method not only requires a large number of negative samples for training, but also the detection accuracy of the infrared image model cannot be guaranteed, and there is a problem that the recognition ability of heating defects of power equipment is not ideal.

[0004] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present application provide a method, device, non-volatile storage medium and electronic device for detecting heating defects, so as to at least solve the technical problem of unsatisfactory detection efficiency of heating defects in related technologies.

[0006] According to one aspect of the embodiments of the present application, a method for detecting heating defects is provided, including: extracting edge features from an infrared image including a power device to obtain an edge contour; extracting infrared features from the infrared image to obtain a feature vector; using a hash function to perform dimensionality reduction mapping on the feature vector to obtain a target hash code; extracting temperature features based on the first pixel points included inside the edge contour; and determining a detection result of the heating defect of the power device based on the target hash code and the temperature features.

[0007] Optionally, the method further includes: acquiring an initial image of the power equipment; processing a first region in the initial image using a mask template to determine the median value of the pixels of a plurality of second pixel points included in the mask template; determining weight values corresponding to other points except the center point among the plurality of second pixel points based on the median value of the pixels, where the center point is the pixel point at the center of the mask template; updating the initial pixel value of the center point based on the weight values and pixel values corresponding to the other points to obtain the target pixel value of the center point, as the filtering process for the first region; and performing the filtering process for the first region in each of a plurality of regions included in the initial image using the mask template to obtain the infrared image.

[0008] Optionally, performing the filtering process for the first region in each of a plurality of regions included in the initial image using the mask template to obtain the infrared image includes: performing the filtering process for each of a plurality of regions included in the initial image using the mask template to obtain a candidate image; determining the signal-to-noise ratio, peak signal-to-noise ratio, and structural similarity of the candidate image, where the signal-to-noise ratio represents the quality of the filtering process for the candidate image, the peak signal-to-noise ratio represents the error based on the pixel points included in the candidate image, and the structural similarity represents the similarity of the candidate image before and after the filtering process; determining a quality score of the candidate image based on the signal-to-noise ratio, the peak signal-to-noise ratio, and the structural similarity; and determining the candidate image as the infrared image when the quality score meets a predetermined quality requirement.

[0009] Optionally, extracting edge features from the infrared image including the power equipment to obtain an edge contour includes: determining the gradient magnitude and direction angle corresponding to each of a plurality of third pixel points included in the infrared image, where the gradient magnitude represents the magnitude of the gray level change of the infrared image, and the direction angle represents the direction of the gray level change in the infrared image; determining a first type of edge points corresponding to the third pixel points with a gradient magnitude greater than a first predetermined threshold, and a second type of edge points corresponding to the third pixel points with a gradient magnitude greater than a second predetermined threshold and less than or equal to the first predetermined threshold; screening the second type of edge points based on the connection relationship between the second type of edge points and the first type of edge points to obtain the screened second type of edge points; and determining the edge contour based on the screened second type of edge points and the first type of edge points.

[0010] Optionally, using a hash function to perform dimensionality reduction mapping on the feature vector to obtain a target hash code includes: using a training hash code to train an initial model to obtain a target model, where the initial model is provided with a first penalty term and a second penalty term for iteration, the first penalty term is used to control the probability of each bit in the training hash code taking an evenly distributed value, and the second penalty term is used to control the probability of each bit in the training hash code generating a non-binary value; using a hash function, and performing dimensionality reduction mapping on the feature vector by the target model to obtain the target hash code.

[0011] Optionally, determining the heat defect detection result of the power equipment based on the target hash code and the temperature feature includes: using the Hamming distance to compare the similarity between the target hash code and a predetermined hash code to determine a similarity comparison result, where the predetermined hash code is the hash code of an infrared image with a heat defect; determining the heat defect detection result based on the similarity comparison result and the temperature feature.

[0012] Optionally, the power equipment includes any one of the following: lightning arrester, current transformer, voltage transformer, coupling capacitor, high-voltage bushing, insulator, and the infrared image is acquired by an inspection robot or an infrared camera.

[0013] According to another aspect of the embodiments of the present application, there is provided a heat defect detection device, including: an edge contour determination module for extracting edge features from an infrared image including a power equipment to obtain an edge contour; an infrared feature extraction module for extracting infrared features from the infrared image to obtain a feature vector; a dimensionality reduction mapping module for using a hash function to perform dimensionality reduction mapping on the feature vector to obtain a target hash code; a temperature feature extraction module for extracting a temperature feature based on a first pixel point included inside the edge contour; and a detection result determination module for determining the heat defect detection result of the power equipment based on the target hash code and the temperature feature.

[0014] According to another aspect of the embodiments of the present application, there is provided a non-volatile storage medium storing multiple instructions, and the instructions are suitable for being loaded and executed by a processor to perform any one of the heat defect detection methods.

[0015] According to another aspect of the embodiments of the present application, there is provided an electronic device, including: one or more processors and a memory, where the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement any one of the heat defect detection methods.

[0016] In an embodiment of the present application, by performing edge feature extraction on an infrared image including power equipment, an edge contour is obtained; infrared features are extracted from the infrared image to obtain a feature vector; a hash function is used to perform dimensionality reduction mapping on the feature vector to obtain a target hash code; temperature extraction is performed based on the first pixel points included inside the edge contour to obtain a temperature feature; based on the target hash code and the temperature feature, a detection result of the heating defect of the power equipment is determined. The purpose of fusing the hash code obtained by dimensionality reduction features with the temperature feature obtained based on the edge contour is achieved, the technical effect of improving the detection efficiency of defects is realized, and furthermore, the technical problem of unsatisfactory detection efficiency of heating defects existing in the related art is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0018] Figure 1 is a flowchart of an optional method for detecting heating defects provided according to an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of an optional method for detecting heating defects provided according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of an optional device for detecting heating defects provided according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0022] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0023] For the convenience of description, some nouns or terms related to the embodiments of this application are described below:

[0024] Deep Hashing is an image retrieval and similarity matching method that combines deep learning technology. It extracts high-dimensional features of images through a deep neural network and maps them into a low-dimensional binary hash code space for quickly retrieving and matching large-scale image datasets. Deep Hashing can maintain the similarity relationship between images while greatly reducing storage and computing requirements.

[0025] Hash Code is a fixed-length code, usually generated by a hash function.

[0026] Hamming Distance is used to measure the number of different characters at corresponding positions in two equal-length strings. In the context of deep hashing, Hamming Distance can be used to calculate the difference between two hash codes. The smaller the distance, the more similar they are.

[0027] The Canny algorithm is a multi-level edge detection algorithm used to accurately and reliably detect and locate edges in an image.

[0028] The Sobel operator is a commonly used edge detection operator in image processing. It locates edges by calculating the gradients of the image in the horizontal and vertical directions.

[0029] According to the embodiments of this application, a method embodiment for detecting heating defects is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described here can be executed in a different order than here.

[0030] Figure 1 is a flowchart of the method for detecting heating defects according to the embodiments of this application, asFigure 1 As shown, the method includes the following steps:

[0031] Step S102, extract edge features from the infrared image including the power equipment to obtain an edge contour;

[0032] It can be understood that by processing the infrared image and extracting the edge contour of the power equipment, the background noise is removed by locating the boundary of the battery equipment, ensuring that subsequent analysis focuses on the key area of the equipment.

[0033] In an optional embodiment, the power equipment includes any one of the following: lightning arrester, current transformer, voltage transformer, coupling capacitor, high-voltage bushing, insulator, and the infrared image is collected by an inspection robot or an infrared camera.

[0034] It can be understood that there can be various types of power equipment suitable for detection, including lightning arresters, current transformers, voltage transformers, coupling capacitors, high-voltage bushings, insulators, etc. The above types of power equipment play key roles in the power system. For example, lightning arresters are used to prevent lightning overvoltage from damaging power equipment, current transformers and voltage transformers are used to measure and monitor the current and voltage of power lines, coupling capacitors are used for signal coupling and filtering, and high-voltage bushings and insulators are used for electrical isolation and support. Since the above power equipment may have heating defects during operation, implementing infrared detection on them is an important means to ensure the safe operation of the power system.

[0035] The inspection robot or camera collects the infrared spectrum file to be detected. There can be various power equipment, such as lightning arresters, current transformers, voltage transformers, coupling capacitors, high-voltage bushings, insulators, etc. The heating defects can include two types: current-induced heating type and voltage-induced heating type. The data format of the infrared spectrum file follows the infrared general data file format. The file format is preferably JPGE, the resolution is preferably 640*380, and the infrared spectrum file data includes parameters such as infrared screenshot file, temperature dot matrix width, temperature dot matrix height, infrared temperature value dot matrix data, infrared data start offset address, emissivity, reflected temperature, ambient temperature, shooting distance, file version, end identifier, etc.

[0036] The defects of current - induced heating type and voltage - induced heating type are mainly distinguished according to the working principles and common fault modes of power equipment. The following is the classification of several power equipment such as lightning arresters, current transformers, voltage transformers, coupling capacitors, high - voltage bushings, and insulators according to the types of defects they may generate. Power equipment that generates defects of current - induced heating type includes current transformers, lightning arresters, and high - voltage bushings. Current transformers work mainly by inducing a proportional current on the secondary side through the primary - side current. If the joint is poorly contacted, the winding is short - circuited, or the resistance increases, it will cause an increase in local current, thus generating heat. This type of defect belongs to the current - induced heating type. In a lightning arrester, the varistor or joint may have an increased contact resistance during long - term operation or under over - voltage conditions, resulting in abnormal heat generation under normal current, which belongs to the current - induced heating type. Faults such as poor contact between the internal conductor and the bushing of a high - voltage bushing and over - current of the conductor will also cause defects of the current - induced heating type.

[0037] Power equipment that generates defects of voltage - induced heating type includes: voltage transformers, coupling capacitors, and insulators. Voltage transformers work by inducing a proportional voltage on the secondary side through the primary - side voltage. If the insulating material ages or the internal electric - field distribution is abnormal, it will cause an increase in dielectric loss and generate heat. This type of defect belongs to the voltage - induced heating type. Coupling capacitors are mainly used for signal coupling and electrical isolation. When the capacitor dielectric deteriorates or the electric - field distribution is uneven, it will cause an increase in dielectric loss and overheating, which belongs to the voltage - induced heating type. For an insulator under high voltage, if local discharge or abnormal electric - field distribution is caused by surface contamination, internal air gaps, etc., it will generate defects of the voltage - induced heating type.

[0038] In an alternative embodiment, the method further includes: performing image acquisition on the power equipment to obtain an initial image; processing a first region in the initial image using a mask template to determine the pixel median value of a plurality of second pixel points included in the mask template; based on the pixel median value, determining the weight values corresponding to other points except the center point among the plurality of second pixel points, where the center point is the pixel point at the center of the mask template; based on the weight values corresponding to other points and the pixel values, updating the initial pixel value of the center point to obtain the target pixel value of the center point, which is used as the filtering process for the first region; using the filtering process for the first region, performing filtering processing on each of the plurality of regions included in the initial image using the mask template to obtain an infrared image.

[0039] It is understandable that the infrared images of power equipment are preprocessed, and an improved mask filtering technique is adopted to reduce image noise while retaining key image information. The infrared images of power equipment can be automatically collected by inspection robots or infrared cameras. The infrared images carry the surface temperature information of the equipment and are the basis for subsequent defect detection. Applying a mask template to the collected initial image, the mask template covers the first area on the image, such as a local part of the equipment. By calculating the pixel median of multiple second pixel points (i.e., the pixel points covered by the mask) within the template, a benchmark is provided for subsequent weight calculation. Based on the pixel median, the weight values corresponding to other points in the mask template except the center point (the pixel point at the center of the mask template) are determined. The principle of calculating the weight value is that the smaller the difference between the pixel value and the median, the greater the weight, and vice versa, which means that the pixel points with a temperature similar to that of the center point will obtain a higher weight. According to the weight values and pixel values of other points, the initial pixel value of the center point is updated to obtain the target pixel value of the center point. This process is essentially a filtering process. By replacing the original pixel value of the center point, the filtering process of the first area is realized, effectively reducing the influence of noise. Repeating the above filtering process of the mask template and applying it to multiple areas included in the initial image, the finally obtained infrared image is globally denoised. The improved mask filtering technique can significantly reduce the background noise in the infrared image and the non-uniformity on the surface of the power equipment, improve the image quality, provide a clearer data basis for subsequent feature extraction and defect diagnosis, avoid the loss of edge information during noise removal by traditional filtering methods, and at the same time improve the operation speed.

[0040] Optionally, the infrared image is preprocessed based on an improved mask filtering method. Considering the scenario where the power equipment is located in a substation, since there are many interference factors in the infrared images obtained from substation inspections, such as background noise, detector noise, etc., these noises need to be removed through a filtering algorithm. Traditional filtering methods can include median filtering, mean filtering, Gaussian filtering, bilateral filtering, etc. The present invention adopts an improved mask filtering method to treat all pixels equally without considering the interaction between pixels to achieve the purpose of arithmetic averaging. This filtering method can not only avoid the loss of edge information during noise removal by traditional filtering methods but also improve the operation speed.

[0041] The mask selection method has 9 different-shaped windows. Taking the central pixel f(x,y) as the reference point within the window, the average value and variance within each window are calculated respectively, and the shielding window with the smallest variance is used for averaging. The mean M of the mask i' is calculated by the formula The variance σ i' is calculated by the formula Where i' is the mask template serial number, and the range of the mask template serial number is 1 to 9. f(x, y) is the pixel value of the mask template, N1 is the size of the mask template, and (x, y) represents the pixel point.

[0042] The improved mask filtering method is to calculate the weights of the pixel points in the template area to further improve the filtering effect. If the median value of the pixels in the mask template is R, the weights of the pixel points in the mask template except the central pixel point can be expressed as r(x, y):

[0043]

[0044] Where abs represents the absolute value. It can be seen that the smaller the difference between the pixel value of any pixel point in the mask template and the median value of the pixels, the more likely it is that it has the same attribute as the central point, and its contribution value is greater. The weight value obtained by calculation is larger, and the filtering effect is similar to median filtering. If the pixel values in the template are evenly distributed, the calculated weight values do not differ much, and the filtering effect is similar to mean filtering. After calculating the weights of the pixel points in the algorithm template, the weighted p(x, y) (i.e., the target pixel value) can be used to replace the previous central point pixel value (i.e., the initial pixel value). The target pixel value can be calculated in the following way. In

[0045] In an alternative embodiment, mask templates are respectively used to perform filtering processing on multiple regions included in the initial image to obtain an infrared image, including: performing filtering processing on multiple regions included in the initial image respectively using mask templates to obtain a candidate image; determining the signal-to-noise ratio, peak signal-to-noise ratio, and structural similarity of the candidate image, where the signal-to-noise ratio represents the quality of filtering of the candidate image, the peak signal-to-noise ratio represents the error based on the pixel points included in the candidate image, and the structural similarity represents the similarity of the candidate image before and after filtering processing; determining the quality score of the candidate image based on the signal-to-noise ratio, peak signal-to-noise ratio, and structural similarity; and determining the candidate image as an infrared image when the quality score meets a predetermined quality requirement.

[0046] It can be understood that the quality control process for preprocessing the infrared images of power equipment ensures that the filtered infrared images have sufficient clarity and accuracy to support subsequent detection of heating defects. Multiple regions in the initial infrared image are filtered separately using a mask template to obtain a series of candidate images, aiming to reduce the noise and non-uniformity in the image and improve the visibility of thermal image details. For each candidate image, its signal-to-noise ratio (SNR), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) are calculated. The signal-to-noise ratio measures the ratio of the image signal to the noise, the peak signal-to-noise ratio evaluates the pixel error between the image and the original image, and the structural similarity evaluates the similarity between images from three aspects: brightness, contrast, and structural features. These three indicators together reflect the clarity and information retention of the image. Based on the signal-to-noise ratio, peak signal-to-noise ratio, and structural similarity, a quality score is determined for each candidate image. This score reflects the overall quality of the image after preprocessing and is an important basis for subsequent determination of whether the image is usable. Only when the quality score of the candidate image meets the predetermined quality requirements will the candidate image be officially determined as the infrared image for subsequent processing.

[0047] Through the above processing, it is ensured that the infrared images used for detection and analysis have sufficient signal-to-noise ratio, small pixel error, and high structural similarity with the original image, thereby improving the accuracy and reliability of subsequent processing and reducing misjudgments or missed judgments caused by poor image quality.

[0048] Optionally, quality assessment is performed on the filtered infrared image. The main evaluation indicators include signal-to-noise ratio (SNR), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM). The signal-to-noise ratio (SNR) is used to measure the ratio of the signal intensity to the noise intensity in an image. It is a dimensionless ratio, usually expressed in decibels (dB). In image processing, a higher SNR indicates better image quality and less noise interference. This indicator helps to evaluate the effect of image preprocessing techniques (such as filtering) in removing background noise. Especially in infrared images, a high SNR means that the thermal image features of the device can be identified more clearly, reducing the false positive rate caused by noise. The peak signal-to-noise ratio (PSNR) is a quantitative indicator to measure the distortion of an image before and after processing such as image compression and filtering. It is calculated based on the mean squared error (MSE) and is expressed in dB. The higher the PSNR, the smaller the difference between the original image and the processed image, that is, the lower the distortion caused by image compression or filtering. In infrared image preprocessing, PSNR can be used to evaluate the degree of detail retention of the filtering algorithm, ensuring that image features are not lost due to over-filtering. The structural similarity (SSIM) comprehensively considers three aspects of the image: structure, brightness, and contrast, and is used to evaluate the visual similarity between two images. The SSIM value ranges from -1 to 1, and the value closer to 1 indicates that the images are more similar. In infrared image processing, SSIM can evaluate whether the filtering algorithm maintains the original thermal map structure and details of the device while removing noise, ensuring that subsequent defect detection is not affected by changes in the image structure.

[0049] The numerical value of the signal-to-noise ratio (SNR) represents the quality of image filtering and can be characterized as follows:

[0050]

[0051] where M and N represent the size of the image, the pixel points of the original image are f(x, y), and the pixel points of the candidate image after denoising are

[0052] The peak signal-to-noise ratio (PSNR) is calculated based on the error of pixel points. where n is the number of bits per pixel, usually taking the value of 8. The structural similarity (SSIM) is measured from three aspects: brightness, contrast, and structural characteristics. Brightness Contrast Structural characteristics where C 1 , C 2 , C 3 are correlation coefficients. The original image is denoted as p1 and the candidate image is denoted as p2. The mean of the original image is μ p1 , and the variance is σ p1 , and the mean of the candidate image is μ p2, with a variance of σ p2 , with a covariance of σ p1p2 .

[0053]

[0054] In an alternative embodiment, edge feature extraction is performed on an infrared image including a power device to obtain an edge contour, including: determining the gradient magnitude and direction angle corresponding to each of a plurality of third pixel points included in the infrared image, where the gradient magnitude represents the magnitude of the grayscale change in the infrared image, and the direction angle represents the direction of the grayscale change in the infrared image; determining a first type of edge points among the plurality of third pixel points whose corresponding gradient magnitude is greater than a first predetermined threshold, and a second type of edge points whose corresponding gradient magnitude is greater than a second predetermined threshold and less than or equal to the first predetermined threshold; screening the second type of edge points based on the connection relationship between the second type of edge points and the first type of edge points to obtain the screened second type of edge points; and determining the edge contour based on the screened second type of edge points and the first type of edge points.

[0055] It can be understood that the gradient magnitude and direction angle of each pixel point in the infrared image are calculated. The gradient magnitude reflects the severity of the grayscale change in the image, that is, the intensity of the edge, and the direction angle indicates the direction of the grayscale change, which helps to locate the direction of the edge. The goal is to quantify the edge information of each pixel point in the image to provide a basis for subsequent edge point classification. The calculated gradient magnitude is compared with two predetermined thresholds, and the pixel points are divided into two categories: the first type of edge points (i.e., strong edge points), whose gradient magnitude is greater than the higher first predetermined threshold, and these points are usually located on the prominent edges in the image; the second type of edge points (i.e., weak edge points), whose gradient magnitude is greater than the lower second predetermined threshold but does not exceed the first threshold, and these points may be part of the edge or the result of noise or color change. Screening is performed based on the connection relationship between the weak edge points and the strong edge points. If a weak edge point is connected to at least one strong edge point, then it may be part of the real edge and can be retained; if a weak edge point is isolated and not connected to a strong edge point, then it may be noise and should be eliminated, taking advantage of the coherence of the edge to help distinguish real edges from false edge points caused by noise.

[0056] The screened weak edge points and all the retained strong edge points are combined to form a complete edge contour, ensuring the continuity and integrity of the edge contour. Even in places where the grayscale change is not very drastic, the retention of weak edge points makes the entire contour clearer and more accurate.

[0057] Optionally, edge feature extraction can be performed on an image that meets the quality requirements (i.e., an infrared image) based on the Canny edge detection algorithm. The Canny algorithm has characteristics such as accurate positioning and strong anti-noise ability. The Canny edge detection algorithm first uses the Sobel operator to calculate the gradient magnitude and direction angle of the image. The magnitude of the gradient represents the degree of change in the image grayscale, and the direction of the gradient represents the direction of the change in the image grayscale. The gradient magnitude represents the size of the gradient and is used to represent the severity of the change in the image grayscale. The gradient magnitude U(x,y) = |g x (x,y)| + |g y (x,y)|, and the direction angle of the gradient where g x (x,y) represents the gradient component in the x direction, and g y (x,y) represents the gradient direction in the y direction. Then, the NMS algorithm is used for local maximum search to suppress non-maximum values and eliminate false values in edge detection. The edges are finally determined through the strong and weak edge point thresholds. Strong edge points are pixel points whose gradient value (i.e., gradient magnitude) is greater than the high threshold (i.e., the first predetermined threshold), and the pixel value is set to 255; weak edge points are pixel points whose gradient value is greater than the low threshold (i.e., the second predetermined threshold) but less than the high threshold, and the pixel value remains unchanged; if the gradient value is less than the low threshold, it will be suppressed and the pixel value is set to 0. Strong edge points can be considered as real edges, while weak edge points may be real edges or may be caused by noise or color changes. Weak edge points caused by real edges are connected to strong edge points, while weak edge points caused by noise are not. The hysteresis connection in the Canny operator means connecting weak edge points to strong edge points to reduce edge breaks.

[0058] Step S104, perform infrared feature extraction on the infrared image to obtain a feature vector;

[0059] It can be understood that deep feature extraction is performed on the infrared image to obtain a high-dimensional feature vector that can describe the shape, structure, and temperature difference distribution of the power equipment. By capturing the subtle differences in the infrared image, rich feature information is generated.

[0060] Optionally, a VGG16 convolutional neural network can be used for infrared image feature extraction. The VGG16 model constructs a deep network by stacking multiple smaller convolutional layers and pooling layers to enhance the model's expressive power. The "16" in the VGG16 network refers to 16 weight layers in the network (13 convolutional layers and 3 fully connected layers). The convolutional layers are mainly used to extract features of the input image, and the fully connected layers are mainly used to map the extracted features to class probabilities. The convolution in VGG16 uses smaller convolutional layers (the size of the convolutional kernel in each layer is fixed at 3x3), and the convolution operation with a stride of 1. A 2x2 max pooling layer is used between every two convolutional layers to reduce the size of the feature map and retain the most significant features. After the last convolutional layer, VGG16 uses three fully connected layers, each with 4096 hidden units, and the last fully connected layer outputs the prediction result of the model. This design enables the network to capture various scale features in the image, and at the same time reduces the number of parameters, making the model easier to train.

[0061] In the infrared image recognition and defect detection of power equipment, the VGG16 network can effectively extract features from the image, such as the shape, color, and texture of the equipment. These features are crucial for identifying the type and status of the equipment and detecting potential heating defects. By training the VGG16 network to learn the feature representation of heating defects in infrared images, automatic analysis and defect recognition of infrared images of power equipment can be achieved, improving the efficiency and accuracy of power equipment maintenance and fault detection.

[0062] First, preprocess the images in the image library, perform image enhancement and normalize the size to 224×224. Each convolutional layer contains a convolution operation and a ReLU activation function. The size of the convolutional kernel is 3×3, the convolution stride is set to 1, and the padding of each layer is set to remain unchanged to ensure that features are not lost. Padding is a technique of adding extra pixels (usually zeros) to the edges of the input image or feature map. Its main purpose is to keep the size of the output feature map consistent with the size of the input image or control the size of the output feature map to meet specific requirements. In image processing, the use of padding can ensure that important feature information is not lost due to the cropping of pixels at the edges during the convolution operation. The pooling layer uses the max pooling operation, the size of the pooling kernel is 2×2, and the stride is set to 2. During the feature extraction process, the output of each layer is obtained after the features extracted from the previous layer pass through convolution, pooling, and the activation function. The output formula of the convolutional layer is h i,j = ReLU((ω k ×X) ij+b), where X is the input image matrix, ω is the convolutional kernel parameter, b is the bias, ReLU is the activation function, and i, j are the rows and columns in the image matrix. The length of the image output matrix of the i-th layer width can be expressed as follows:

[0063]

[0064] where f i is the size of the convolutional kernel of the i-th layer, p i is the padding data of the i-th layer, s i is the stride of the i-th layer, is the number of convolutional kernels of the i-th layer. To prevent overfitting and improve the generalization ability of the model, a dropout structure is adopted in the fully connected layer. The feature dimension extracted by the final VGG16 network is 4096. Using 4096-dimensional image features for feature similarity calculation will result in an overly large computational amount. By adding a hidden layer between the convolutional layer and the output layer and using the sigmoid activation function in the hidden layer, the output value of the feature vector is placed between 0 and 1, and the output is transformed into a binary vector represented by 01 as the binary retrieval vector. The obtained 4096-dimensional feature vector is reduced to a 128-dimensional binary retrieval vector (i.e., the above-mentioned feature vector).

[0065] Step S106, using a hash function to perform dimensionality reduction mapping on the feature vector to obtain the target hash code;

[0066] It can be understood that by using a hash function to perform dimensionality reduction mapping on the extracted high-dimensional feature vector, a target hash code with a fixed length is generated. Dimensionality reduction processing can greatly reduce the storage and processing requirements of data, while maintaining the similarity relationship between images, enabling fast retrieval and matching of similar infrared images even in the face of a large-scale image library.

[0067] In an optional embodiment, using a hash function to perform dimensionality reduction mapping on the feature vector to obtain the target hash code includes: using the training hash code to train the initial model to obtain the target model, where the initial model is provided with a first penalty term and a second penalty term for iteration, the first penalty term is used to control the probability of each bit in the training hash code taking an equal value, and the second penalty term is used to control the probability of each bit in the training hash code generating a non-binary value; using a hash function, the target model performs dimensionality reduction mapping on the feature vector to obtain the target hash code.

[0068] It can be understood that the deep hashing technique is used to reduce the dimension and binarize the high-dimensional feature vectors extracted from the infrared images to generate the target hash codes. In this process, specific penalty terms are used to optimize the generation of hash codes. First, an initial model is defined, which includes the feature extraction part of the convolutional neural network (CNN) and a hash function for generating hash codes. During the model training process, two penalty terms are introduced: the first penalty term and the second penalty term. The first penalty term is used to control the probability that each bit in the generated hash code takes 0 or 1 during the training process, ensuring that this probability is close to 50%, that is, the hash code distribution is as uniform as possible. This is beneficial to the performance of the hash code in retrieval and comparison because the uniformly distributed hash code can more effectively represent the features of the image, avoiding the situation where some bits are overly active while other bits are almost inactive, and improving the discrimination of the hash code. The second penalty term is used to control the probability of non-binarized values generated by each bit in the hash code, that is, to ensure that each dimension of the hash code is 0 or 1. Since the hash code is designed to be binary, non-binarized values will interfere with the generation of the hash code and reduce the accuracy and efficiency of subsequent processing (such as Hamming distance calculation). During the training process, the model parameters are iteratively optimized to minimize the loss function (including classification loss and other losses), while controlling the first penalty term and the second penalty term to generate high-quality target hash codes. Enable the initial model to generate a hash code from the input infrared image feature vectors that can both reflect the image features and have the characteristics of binarization and uniform distribution. This hash code has a lower dimension for easy storage and fast retrieval, and at the same time, through the control of the penalty terms, each of its bits has the characteristics of binarization (0 or 1) and uniform distribution.

[0069] Optionally, based on the deep hashing algorithm, the feature images extracted by the convolutional neural network are mapped into hash codes through a hashing method, and at the same time, the edge contours are superimposed to extract the temperature of the target area and calibrate the range of the heating interval to form a diagnosis model for heating defects based on deep hashing (i.e., the above-mentioned target model). During the training process, the number of infrared maps of heating defects is n, and the dimension of the feature vector of each image is d. After the extraction of the image feature values, the data set R' d×n represents a real number matrix of dimension d×n, x n represents the feature vector of the nth image, k is the set of binary codes H∈{-1,1} k×n and the nth column H∈{-1,1} k represents the binary code of the feature x n of the nth image. For the infrared map x n the feature space can be represented by a set of hash functions H = {h 1 ,h 2 ,......,h n}, and each hash function encodes x n as hk (x n ), the k-bit encoding of x n is represented by y n , and y n =H(x n )={h 1 (x n ), h 2 (x n ),...., h k (x n )}. Project y n into a matrix, and the projection matrix is intercept vector Then while the hash code can preferably reflect the high-level semantics of the image, the distribution of the hash code itself should also be more uniform. Therefore, the probability that any bit in the hash code is 0 or 1 is 50%. To achieve this goal, add as a penalty term (i.e., the first penalty term) to the loss function. If this term is added to the loss function alone as a penalty term, the binary code output may fluctuate around 0.5, resulting in a large error in subsequent hash encoding. To make the output of all neurons as close to 0 or 1 as possible, supplement as a penalty term (i.e., the second penalty term), and the new term is to obtain the objective function of the neural network as where L(y n , y n ) is the loss function, w is the weight of the network, λ is the description of the importance of the weight, ||w|| is the penalty term to make the weight lightweight, and e H is the bias of the fully connected layer

[0070] Step S108: Extract the temperature based on the first pixel points included inside the edge contour to obtain temperature features;

[0071] It can be understood that according to the extracted edge contour, the temperature value of each pixel point included in the contour is read to generate temperature features. Using the temperature data of the infrared image provides intuitive and accurate information for locating the key area of the heating defect.

[0072] Optionally, for the temperature similarity detection of the infrared spectrum, it is detected by the corresponding temperature values of the pixels within the contour area. The image pixel is p(x, y), and preferably the range of x is 0 - 640, and the range of y is 0 - 380; the temperature T xy corresponding to the pixel. Determine whether it is a heating defect according to the calibrated abnormal heating interval range within the contour area.

[0073] Step S110: Determine the detection result of the heating defect of the power equipment based on the target hash code and the temperature feature.

[0074] It can be understood that by combining the target hash code and the temperature feature, the Hamming distance and the temperature similarity algorithm are used for comprehensive analysis to determine the detection result of the heating defect of the power equipment. The hash code helps to quickly identify the visual similarity between images, while the temperature feature provides direct evidence of the heating state. The combination of the two makes the detection of heating defects more accurate and efficient.

[0075] In an optional embodiment, determining the detection result of the heating defect of the power equipment based on the target hash code and the temperature feature includes: using the Hamming distance to compare the similarity between the target hash code and the predetermined hash code, and determining the similarity comparison result, where the predetermined hash code is the hash code of the infrared image with a heating defect; based on the similarity comparison result and the temperature feature, determining the detection result of the heating defect.

[0076] It can be understood that converting the hash code to a binary code containing only 0 and 1 simplifies the storage and retrieval process. The Hamming distance is used as a measure of similarity to compare the similarity between the target hash code and the predetermined hash code. The Hamming distance refers to the number of different characters at the corresponding positions of two equal-length strings (in this embodiment, the hash codes). The Hamming distance is the number of different bits between the two codes. The predetermined hash code is generated from the infrared image known to have a heating defect and can be regarded as a representation of the "thermal defect pattern". By calculating the Hamming distance between the target hash code and these patterns, the similarity between the newly detected infrared image and the thermal defect pattern can be judged. If the Hamming distance is small, it means that the two hash codes are very similar, and it is initially judged that the equipment in the image may have a heating defect.

[0077] Optionally, based on the heating defect infrared diagnosis model (i.e., the target model), the Hamming distance and the temperature similarity algorithm are used for similarity judgment. The formula for calculating image similarity is:

[0078] b n =sgn(W T φ(y,θ))

[0079]

[0080] where W represents the network weight, T represents the image feature mapping matrix, φ(y n ; θ) represents the output of the input image passing through the fully connected layer, b n represents the binary form of the hash code, δ nIt represents the feature representation after feature extraction of the input image. While calculating the similarity of the image before, the probability value of the infrared spectrum to be detected belonging to the image matching the training set is calculated. The probability formula is as follows:

[0081] Among them, k represents the length of the hash code, C represents the category of the mixed features (including current heating type and voltage heating type), h i represents the function to be detected, represents the hash function of the image matching the training set similar to the image to be retrieved, represents the hash function of other types of images except the images matching the training set similar to the infrared spectrum to be detected. For the temperature similarity detection of the infrared spectrum, it is detected by the temperature values corresponding to the pixels within the contour area. The image pixels are p(x,y), generally the range of x is 0 - 640, and the range of y is 0 - 380; the temperature T corresponding to the pixel xy . According to the calibrated abnormal heating interval range within the contour area, it is determined whether there is a heating defect to obtain the heating defect detection result.

[0082] Through the above step S102, edge feature extraction is performed on the infrared image including the power equipment to obtain the edge contour; in step S104, infrared feature extraction is performed on the infrared image to obtain the feature vector; in step S106, the hash function is used to perform dimensionality reduction mapping on the feature vector to obtain the target hash code; in step S108, temperature extraction is performed based on the first pixel points included inside the edge contour to obtain the temperature feature; in step S110, based on the target hash code and the temperature feature, the heating defect detection result of the power equipment is determined. It can achieve the purpose of fusing the hash code obtained by dimensionality reduction features with the temperature feature obtained based on the edge contour, improve the accuracy of defect detection, achieve the technical effect of improving the detection efficiency of heating defects, and further solve the technical problem of unsatisfactory detection efficiency of heating defects existing in the related technology.

[0083] Based on the above embodiments and optional embodiments, the present application proposes an optional implementation manner, Figure 2 which is a schematic diagram of an optional heating defect detection method provided according to the embodiments of the present application, as Figure 2 shown and will be described below.

[0084] Step S1, the inspection robot or camera collects the infrared spectrum file to be detected. There can be various types of power equipment suitable for detection, including lightning arresters, current transformers, voltage transformers, coupling capacitors, high-voltage bushings, insulators, etc. The above types of power equipment play key roles in the power system. For example, lightning arresters are used to prevent lightning overvoltage from damaging power equipment, current transformers and voltage transformers are used to measure and monitor the current and voltage of power lines, coupling capacitors are used for signal coupling and filtering, and high-voltage bushings and insulators are used for electrical isolation and support. Since the above power equipment may have heating defects during operation, infrared detection of them is an important means to ensure the safe operation of the power system.

[0085] Step S2, preprocess the infrared image based on the improved mask filtering method. Considering that the scene where the power equipment is located is a substation, there are many interference factors in the infrared images obtained from substation inspections, such as background noise, detector noise, etc. These noises need to be removed through a filtering algorithm. Traditional filtering methods can use median filtering, mean filtering, Gaussian filtering, bilateral filtering, etc. The present invention adopts an improved mask filtering method, which treats all pixels equally without considering the interaction between pixels to achieve the purpose of arithmetic mean. This filtering method can not only avoid losing edge information during noise removal by traditional filtering methods but also improve the operation speed. The mask selection method has 9 different-shaped windows. Taking the central pixel f(x, y) as the reference point within the window, calculate the average value and variance within each window respectively, and use the shielding window with the smallest variance for averaging. The mean M i' of the mask is calculated as The variance σ i' is calculated as where i' is the mask template serial number, the range of the mask template serial number is 1 - 9, f(x, y) is the pixel value of the mask template, N1 is the size of the mask template, and (x, y) represents the pixel point.

[0086] The improved mask filtering method replaces arithmetic mean with weighted average. This replacement can adaptively change the weights of pixel points within the template area, further improving the filtering effect. If the median value of the pixels in the mask template is R, the weights of the pixel points except the central pixel point in the mask template can be expressed as r(x, y):

[0087]

[0088] Among them, abs represents the absolute value. It can be seen that the smaller the difference between the pixel value of any pixel point in the mask template and the pixel median, the more likely it is to have the same attribute as the center point, the greater its contribution value, and the greater the calculated weight value. The filtering effect is similar to median filtering. If the pixel values in the template are evenly distributed, the calculated weight values do not differ much, and the filtering effect is similar to mean filtering. After calculating the weights of the pixel points in the algorithm template, the weighted p(x, y) (i.e., the target pixel value) can be used to replace the previous center point pixel value (i.e., the initial pixel value). The target pixel value can be calculated in the following way. In

[0089] Step S3, perform quality evaluation on the filtered infrared image. The main evaluation indicators include signal-to-noise ratio SNR, peak signal-to-noise ratio PSNR, and structural similarity SSIM. The signal-to-noise ratio SNR is used to measure the ratio of the signal intensity to the noise intensity in the image. It is a dimensionless ratio, usually expressed in decibels (dB). In image processing, the higher the SNR, the better the image quality and the smaller the influence of noise. This indicator helps to evaluate the effect of image preprocessing techniques (such as filtering) in removing background noise. Especially in infrared images, a high SNR means that the thermal image features of the device can be recognized more clearly, reducing the false alarm rate caused by noise. The peak signal-to-noise ratio PSNR is a quantitative indicator to measure the image distortion before and after image compression, filtering, etc. It is calculated based on the mean square error (MSE) and is in units of dB. The higher the PSNR, the smaller the difference between the original image and the processed image, that is, the lower the distortion caused by image compression or filtering. In infrared image preprocessing, PSNR can be used to evaluate the degree of detail retention of the filtering algorithm for the image, ensuring that image features will not be lost due to excessive filtering. The structural similarity SSIM comprehensively considers the structure, brightness, and contrast of the image to evaluate the visual similarity between two images. The SSIM value ranges from -1 to 1, and the closer the value is to 1, the more similar the images are. In infrared image processing, SSIM can evaluate whether the filtering algorithm maintains the original thermal map structure and details of the device while removing noise, ensuring that subsequent defect detection will not be affected by changes in the image structure.

[0090] The numerical value of the signal-to-noise ratio SNR represents the quality of image filtering and can be characterized in the following way:

[0091]

[0092] Among them, M and N represent the size of the image. The pixel points of the original image are f(x, y), and the pixel points of the candidate image after denoising are

[0093] The peak signal-to-noise ratio PSNR is calculated based on the error of pixel points. Among them n is the number of bits of the pixel, usually taking the value of 8. The structural similarity SSIM measures from three aspects: brightness, contrast, and structural characteristics. Brightness Contrast Structural characteristics Among them, C 1 , C 2 , C 3 are correlation coefficients. The initial image is denoted as p1 and the candidate image is denoted as p2. The mean of the initial image is μ p1 , and the variance is σ p1 . The mean of the candidate image is μ p2 , and the variance is σ p2 , and the covariance is σ p1p2 .

[0094]

[0095] In step S4, the edge features of the image that meets the quality requirements (i.e., the infrared image) can be extracted based on the Canny edge detection algorithm. The Canny algorithm has the characteristics of accurate positioning and strong anti-noise ability. The Canny edge detection algorithm first uses the Sobel operator to calculate the gradient magnitude and direction angle of the image. The magnitude of the gradient represents the magnitude of the change in the image grayscale, and the direction of the gradient represents the direction of the change in the image grayscale. The gradient magnitude represents the size of the gradient and is used to represent the severity of the change in the image grayscale. The gradient magnitude U(x, y) = |g x (x, y)| + |g y (x, y)|, and the direction angle of the gradient Among them, g x (x, y) represents the gradient component in the x direction, and g y (x, y) represents the gradient direction in the y direction. Then, the NMS algorithm is used for local maximum search to suppress non-maximum values and eliminate false values in edge detection. The edges are finally determined through the strong and weak edge point thresholds. Strong edge points are pixel points whose gradient value (i.e., gradient magnitude) is greater than the high threshold (i.e., the first predetermined threshold), and the pixel value is set to 255; weak edge points are pixel points whose gradient value is greater than the low threshold (i.e., the second predetermined threshold) but less than the high threshold, and the pixel value remains unchanged; if the gradient value is less than the low threshold, it will be suppressed and the pixel value is set to 0. Strong edge points can be considered as real edges, while weak edge points may be real edges or may be caused by noise or color changes. The weak edge points caused by real edges are connected to the strong edge points, while the weak edge points caused by noise are not. The hysteresis connection in the Canny operator refers to connecting weak edge points to strong edge points to reduce edge breaks.

[0096] Step S5: Use a VGG16 convolutional neural network machine to extract infrared image features. First, preprocess the images in the image library, perform image enhancement, and normalize the size to 224×224. Each convolutional layer contains a convolution operation and a ReLU activation function. The size of the convolution kernel is 3×3, the convolution stride is set to 1, and the padding of each layer is set to remain unchanged to ensure that features are not lost. Padding is a technique of adding extra pixels (usually zeros) to the edges of the input image or feature map. Its main purpose is to keep the size of the output feature map consistent with the size of the input image, or to control the size of the output feature map to meet specific requirements. In image processing, the use of padding can ensure that important feature information is not lost due to the cropping of pixels at the edges during the convolution operation. The pooling layer uses max pooling operation, the size of the pooling kernel is 2×2, and the stride is set to 2. During the feature extraction process, the output of each layer is obtained after the features extracted from the previous layer pass through convolution, pooling, and activation functions. The output formula of the convolutional layer is h i,j = ReLU((ω k ×X) ij + b), where X is the input image matrix, ω is the convolution kernel parameter, b is the bias, ReLU is the activation function, and i, j are the rows and columns in the image matrix. The length width of the image output matrix of the i-th layer can be expressed in the following way:

[0097]

[0098] where, f i is the size of the i-th layer convolution kernel, p i is the i-th layer padding data, s i is the i-th layer stride, is the number of convolution kernels of the i-th layer. To prevent overfitting and improve the generalization ability of the model, a dropout structure is adopted in the fully connected layer. The final feature dimension extracted by the VGG16 network is 4096. Using 4096-dimensional image features for feature similarity calculation will result in an overly large computational amount. By adding a hidden layer between the convolutional layer and the output layer, and using the sigmoid activation function in the hidden layer, the output value of the feature vector is placed between 0 and 1, and the output is transformed into a binary vector represented by 01 as the binary retrieval vector. The obtained 4096-dimensional feature vector is reduced to a 128-dimensional binary retrieval vector (i.e., the above-mentioned feature vector).

[0099] Step S6: Based on the deep hashing algorithm, the feature images extracted by the convolutional neural network are mapped into hash codes through the hashing method. At the same time, the temperature of the target area is extracted by superimposing the edge contour, and the fever interval range is calibrated to form a fever defect atlas diagnosis model based on deep hashing (i.e., the above-mentioned target model). During the training process, the number of infrared images of fever defects is n, and the dimension of the feature vector of each image is d. After the image feature values are extracted, for the infrared image x n The feature space can be represented by a set of hash functions H = {h 1 , h 2 ,......, h n}, and each hash function encodes x n as h k (x n ), then y n = H(x n ) = {h 1 (x n ), h 2 (x n ),...., h k (x n )}. Project it into the matrix and the intercept vector then y n = sgn(f(W T x n + b n ). While the hash code can reflect the high-level semantics of the image as well as possible, the distribution of the hash code itself should also be more uniform. Therefore, the probability that any bit in the hash code is 0 or 1 is 50%. To achieve this goal, is added as a penalty term (i.e., the first penalty term) to the loss function. If this term is added to the loss function alone as a penalty term, the binary code output may fluctuate around 0.5, resulting in a large error in subsequent hash encoding. To make the output of all neurons as close to 0 or 1 as possible, is supplemented as a penalty term (i.e., the second penalty term), and the new term is to obtain the objective function of the neural network as

[0100] Step S7: The inspection robot or camera collects the infrared image file to be detected.

[0101] Step S8: Preprocess the infrared image based on the improved mask filtering method. Same as step 2.

[0102] Step S9: Similar to the quality evaluation of the filtered infrared image. Judge whether it meets the quality evaluation threshold requirements. Same as step S3.

[0103] Step S10: Extract edge features from the images meeting the quality requirements based on the Canny edge detection algorithm. Same as step S4.

[0104] Step S11: Based on the infrared diagnosis model for thermal defects (i.e., the target model), use the Hamming distance and temperature similarity algorithm to perform similarity judgment. The formula for image similarity calculation:

[0105] b n =sgn(W T φ(y,θ))

[0106]

[0107] where W represents the network weight, T represents the image feature mapping matrix, φ(y n ; θ) represents the output of the fully connected input image, b n represents the binary form of the hash code, and δ n represents the feature representation of the input image after feature extraction. While calculating the similarity of the image before, calculate the probability value of the infrared spectrum to be detected belonging to the images matching the training set. The probability formula is as follows:

[0108] where k represents the length of the hash code, C represents the category of mixed features (including current thermal type, voltage thermal type), h i represents the function to be detected, represents the hash function of the images matching the training set similar to the image to be retrieved, represents the hash function of other types of images except the images matching the training set similar to the infrared spectrum to be detected. For the temperature similarity detection of the infrared spectrum, detect through the temperature values corresponding to the pixels within the contour area. The image pixels are p(x, y), generally the range of x is 0 - 640, and the range of y is 0 - 380; the temperature T xy corresponding to the pixels. According to the calibrated abnormal heating range within the contour area, determine whether it is a heating defect to obtain the heating defect detection result. After the training and learning of the above steps, in actual detection, form the thermal defect detection of infrared spectrum deep hashing, so as to realize the accurate detection of current thermal type and voltage thermal type heating equipment.

[0109] The above optional embodiments achieve at least the following effects: A diagnostic method based on infrared image deep hashing combined with edge features and temperature data is proposed. Using improved mask filtering preprocessing, the Canny algorithm is used to extract edges. After the deep features of the VGG16 network are extracted, hash coding is performed, effectively reducing the computational complexity and enhancing the robustness of the model. Through the comprehensive judgment of Hamming distance and temperature similarity, the detection accuracy of heating defects is significantly improved, the sample dependence is reduced, the misjudgment rate is lowered, and efficient and accurate diagnosis of thermal defects in infrared spectra is realized.

[0110] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0111] In this embodiment, a heating defect detection device is also provided. This device is used to implement the above embodiments and preferred embodiments, and those that have been described will not be repeated. As used hereinafter, the terms "module" and "device" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0112] According to an embodiment of the present application, an apparatus embodiment for implementing the heating defect detection method is also provided. Figure 3 It is a schematic diagram of a heating defect detection device according to an embodiment of the present application, as Figure 3 shown. The above heating defect detection device includes: an edge contour determination module 302, an infrared feature extraction module 304, a dimensionality reduction mapping module 306, a temperature feature extraction module 308, and a detection result determination module 310. The device will be described below.

[0113] The edge contour determination module 302 is used to extract edge features from an infrared image including a power device to obtain an edge contour.

[0114] The infrared feature extraction module 304 is connected to the edge contour determination module 302 and is used to extract infrared features from the infrared image to obtain a feature vector.

[0115] The dimensionality reduction mapping module 306 is connected to the infrared feature extraction module 304 and is used to perform dimensionality reduction mapping on the feature vector using a hash function to obtain a target hash code.

[0116] The temperature feature extraction module 308 is connected to the dimensionality reduction mapping module 306 and is used to extract temperature based on the first pixel points included inside the edge contour to obtain temperature features.

[0117] A detection result determination module 310, connected to the temperature feature extraction module 308, is configured to determine a heat defect detection result of the power device based on the target hash code and the temperature feature.

[0118] In a heat defect detection device provided by an embodiment of the present application, by setting an edge contour determination module 302 for extracting edge features from an infrared image including a power device to obtain an edge contour; an infrared feature extraction module 304, connected to the edge contour determination module 302, for extracting infrared features from the infrared image to obtain a feature vector; a dimensionality reduction mapping module 306, connected to the infrared feature extraction module 304, for performing dimensionality reduction mapping on the feature vector by using a hash function to obtain a target hash code; a temperature feature extraction module 308, connected to the dimensionality reduction mapping module 306, for extracting temperature based on first pixel points included inside the edge contour to obtain a temperature feature; and a detection result determination module 310, connected to the temperature feature extraction module 308, for determining a heat defect detection result of the power device based on the target hash code and the temperature feature. The purpose of improving the accuracy of defect detection by fusing the hash code obtained from the dimensionality reduction feature with the temperature feature obtained based on the edge contour is achieved, the technical effect of improving the detection efficiency of heat defects is realized, and thus the technical problem of unsatisfactory detection efficiency of heat defects in the related art is solved.

[0119] It should be noted that the above-mentioned various modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following manner: the above-mentioned various modules can be located in the same processor; or, the above-mentioned various modules are located in different processors in any combination.

[0120] Here, it should be noted that the above-mentioned edge contour determination module 302, infrared feature extraction module 304, dimensionality reduction mapping module 306, temperature feature extraction module 308, and detection result determination module 310 correspond to steps S102 to S110 in the embodiment. The examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in the above embodiment. It should be noted that the above modules, as part of the device, can run in a computer terminal.

[0121] It should be noted that the optional or preferred implementation manners of this embodiment can refer to the relevant descriptions in the embodiment, and will not be repeated here.

[0122] The above-mentioned heat defect detection device may further include a processor and a memory. The edge contour determination module 302, infrared feature extraction module 304, dimensionality reduction mapping module 306, temperature feature extraction module 308, detection result determination module 310, etc. are all stored in the memory as program units, and the processor executes the above program units stored in the memory to implement corresponding functions.

[0123] The processor contains a core, which retrieves the corresponding program units from the memory. One or more cores can be set. The memory may include non-permanent memory in a computer-readable medium, forms such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.

[0124] An embodiment of the present application provides a non-volatile storage medium, on which a program is stored, and when the program is executed by a processor, a heat defect detection method is implemented.

[0125] An embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program stored on the memory and executable on the processor. When the processor executes the program, the following steps are implemented: extracting edge features from an infrared image including a power device to obtain an edge contour; extracting infrared features from the infrared image to obtain a feature vector; using a hash function to perform dimensionality reduction mapping on the feature vector to obtain a target hash code; extracting temperature based on the first pixel points included inside the edge contour to obtain a temperature feature; determining a heat defect detection result of the power device based on the target hash code and the temperature feature. The device herein can be a server, a PC, etc.

[0126] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program initialized with the following method steps: extracting edge features from an infrared image including a power device to obtain an edge contour; extracting infrared features from the infrared image to obtain a feature vector; using a hash function to perform dimensionality reduction mapping on the feature vector to obtain a target hash code; extracting temperature based on the first pixel points included inside the edge contour to obtain a temperature feature; determining a heat defect detection result of the power device based on the target hash code and the temperature feature.

[0127] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0129] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including instruction means that implement the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more of the flows Figure 1 one or more of the flows and / or blocks Figure 1 or means for implementing the functions specified in one or more of the blocks.

[0131] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0132] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0133] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information accessible by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0134] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0135] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] The above are only embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A method for detecting heating defects, characterized in that: include: Extract edge features from infrared images including power equipment to obtain edge contours; Extracting infrared features from the infrared image to obtain a feature vector; Using a hash function to perform dimension reduction mapping on the feature vector to obtain a target hash code; Performing temperature extraction based on a first pixel point included in the edge contour to obtain a temperature feature; Based on the target hash code and the temperature characteristic, a heating defect detection result of the electric equipment is determined.

2. The method according to claim 1, characterized in that The method further comprises: Capturing an image of the electric power equipment to obtain an initial image; Processing the first area in the initial image using the mask template to determine the pixel median of a plurality of second pixel points included in the mask template; Based on the pixel median, determine the weight values ​​corresponding to the other points in the plurality of second pixel points except the center point, wherein the center point is the pixel point at the center of the mask template; Based on the weight values ​​and pixel values ​​corresponding to the other points, the initial pixel value of the center point is updated to obtain the target pixel value of the center point as filtering processing for the first area; The first region is subjected to filtering processing, and the plurality of regions included in the initial image are respectively subjected to filtering processing using the mask template to obtain the infrared image.

3. The method according to claim 2, characterized in that The plurality of regions included in the initial image are respectively subjected to filtering processing using the mask template to obtain the infrared image, including: Based on the multiple regions included in the initial image, the mask template is used to perform filtering processing respectively to obtain a candidate image; Determining a signal-to-noise ratio, a peak signal-to-noise ratio, and a structural similarity of the candidate image, wherein the signal-to-noise ratio indicates the quality of filtering the candidate image, the peak signal-to-noise ratio indicates an error based on pixels included in the candidate image, and the structural similarity indicates the similarity of the candidate image before and after filtering; Determining a quality score of the candidate image based on the signal-to-noise ratio, the peak signal-to-noise ratio, and the structural similarity; In a case where the quality score meets a predetermined quality requirement, the candidate image is determined to be the infrared image.

4. The method according to claim 1, characterized in that The step of extracting edge features from the infrared image including the electric power equipment to obtain edge contours includes: Determine the gradient amplitude and direction angle respectively corresponding to a plurality of third pixel points included in the infrared image, wherein the gradient amplitude represents the magnitude of the grayscale change of the infrared image, and the direction angle represents the direction of the grayscale change in the infrared image; Determine, among the plurality of third pixel points, first-category edge points whose corresponding gradient magnitudes are greater than a first predetermined threshold, and second-category edge points whose corresponding gradient magnitudes are greater than a second predetermined threshold and less than or equal to the first predetermined threshold; Based on the connection relationship between the second-category edge points and the first-category edge points, the second-category edge points are screened to obtain screened second-category edge points; The edge contour is determined based on the filtered second-category edge points and the first-category edge points.

5. The method according to claim 1, characterized in that The adopting of a hash function to perform dimension reduction mapping on the feature vector to obtain a target hash code includes: Using a training hash code, an initial model is trained to obtain a target model, wherein the initial model is provided with a first penalty item and a second penalty item for iteration, the first penalty item is used to control the probability of uniform value of each bit in the training hash code, and the second penalty item is used to control the probability of non-binary value generated by each bit in the training hash code; The target model uses a hash function to perform dimension reduction mapping on the feature vector to obtain the target hash code.

6. The method according to claim 1, characterized in that The step of determining the heating defect detection result of the electric power equipment based on the target hash code and the temperature feature includes: Using Hamming distance, comparing the similarity between the target hash code and a predetermined hash code to determine a similarity comparison result, wherein the predetermined hash code is a hash code of an infrared image with a heating defect; Based on the similarity comparison result and the temperature characteristic, the heating defect detection result is determined.

7. The method according to any one of claims 1 to 6, characterized in that: The power equipment includes any one of the following: a lightning arrester, a current transformer, a voltage transformer, a coupling capacitor, a high-voltage bushing, and an insulator. The infrared image is collected by an inspection robot or an infrared camera.

8. A heating defect detection device, characterized in that: include: An edge contour determination module is used to extract edge features from infrared images including power equipment to obtain edge contours; An infrared feature extraction module, used to extract infrared features from the infrared image to obtain a feature vector; A dimension reduction mapping module is used to use a hash function to perform dimension reduction mapping on the feature vector to obtain a target hash code; A temperature feature extraction module, used for performing temperature extraction based on a first pixel point included in the edge contour to obtain a temperature feature; The detection result determination module is used to determine the heating defect detection result of the electric equipment based on the target hash code and the temperature feature.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the heating defect detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the heating defect detection method described in any one of claims 1 to 7.

Images