Intelligent detection method and device for insulator defects, electronic equipment and medium

Abstract

The invention relates to an insulator defect intelligent detection method and device, electronic equipment and a medium, and the method comprises the steps: obtaining an infrared image of a to-be-detected insulator, carrying out the preprocessing of the infrared image, and extracting a feature value, and obtaining an infrared preprocessing image and an infrared image feature value; obtaining a visible light image of a to-be-detected insulator, and performing preprocessing and feature value extraction on the visible light image to obtain a visible light preprocessed image and a visible light image feature value; based on the infrared image feature value and the visible light image feature value, fusing the infrared preprocessed image and the visible light preprocessed image by adopting a gradient pyramid decomposition algorithm to obtain a fused image and position information of the insulator defect; and performing defect identification on the fused image by adopting a pre-trained insulator defect detection deep learning model to obtain an insulator defect grade. According to the invention, the defect grade of the insulator can be identified more accurately, so that the operation safety and maintenance efficiency of a power system are improved.

Description

Technical Field

[0001] The present application relates to the technical field of power system detection and monitoring, and specifically to an intelligent detection method, device, electronic equipment and medium for insulator defects. Background Art

[0002] In the power system, the safe and stable operation of transmission lines is crucial to ensuring social and economic activities and the daily lives of residents. As a key component in the transmission line, the main function of insulators is to support the conductors and prevent the current from returning to the ground, ensuring the safety and reliability of power transmission. However, insulators will be affected by a variety of external factors during long-term operation, such as strong electric fields, ultraviolet radiation, temperature changes, humidity, and pollutant deposition. These factors may cause defects such as dirt, aging, cracks, and corrosion on the surface of the insulator, thereby affecting its insulation performance and increasing the risk of transmission line failure.

[0003] At present, the detection of insulator defects usually adopts visual inspection, electrical testing, ultrasonic testing, laser scanning, or single infrared thermal imaging technology, but these technologies generally have problems such as high cost of detection equipment, need for special equipment and personnel, strict detection conditions, poor real-time performance, and low accuracy of single infrared thermal imaging. Summary of the invention

[0004] In view of the above problems, the embodiments of the present application provide an intelligent detection method, device, electronic device and medium for insulator defects to overcome or partially overcome the shortcomings of the prior art.

[0005] In a first aspect, the present application provides an intelligent detection method for insulator defects, comprising:

[0006] Acquire an infrared image of the insulator to be inspected, perform preprocessing and feature value extraction on the infrared image, and obtain an infrared preprocessing image and infrared image feature values;

[0007] Acquire a visible light image of the insulator to be inspected, perform preprocessing and feature value extraction on the visible light image, and obtain a visible light preprocessing image and visible light image feature values;

[0008] Based on the infrared image feature value and the visible light image feature value, a gradient pyramid decomposition algorithm is used to fuse the infrared pre-processed image and the visible light pre-processed image to obtain a fused image and location information of insulator defects;

[0009] A pre-trained insulator defect detection deep learning model is used to perform defect recognition on the fused image to obtain the insulator defect grade.

[0010] In a second aspect, the present application also provides an intelligent detection device for insulator defects, the device comprising:

[0011] A first acquisition unit is used to acquire an infrared image of the insulator to be detected, preprocess the infrared image and extract feature values ​​to obtain an infrared preprocessed image and infrared image feature values;

[0012] A second acquisition unit is used to acquire a visible light image of the insulator to be inspected, preprocess the visible light image and extract feature values ​​to obtain a visible light preprocessed image and a visible light image feature value;

[0013] a fusion unit, configured to fuse the infrared pre-processed image and the visible light pre-processed image by using a gradient pyramid decomposition algorithm based on the infrared image feature value and the visible light image feature value, so as to obtain a fused image and location information of insulator defects;

[0014] The prediction unit is used to use a pre-trained insulator defect detection deep learning model to perform defect recognition on the fused image to obtain the insulator defect level.

[0015] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes any of the above-mentioned intelligent detection methods for insulator defects.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes any of the above-mentioned intelligent detection methods for insulator defects.

[0017] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0018] The present application obtains an infrared image of the insulator to be detected, preprocesses and extracts feature values ​​of the infrared image, and obtains an infrared preprocessed image and an infrared image feature value; and obtains a visible light image of the insulator to be detected, preprocesses and extracts feature values ​​of the visible light image, and obtains a visible light preprocessed image and a visible light image feature value; then, based on the infrared image feature value and the visible light image feature value, the infrared preprocessed image and the visible light preprocessed image are fused using a gradient pyramid decomposition algorithm to obtain a fused image and the location information of the insulator defect; finally, a pre-trained insulator defect detection deep learning model is used to identify defects on the fused image to obtain the insulator defect level. Compared with the prior art, the present invention uses infrared images and visible light images to intelligently identify the defect level of the insulator, which not only reduces the risk and cost of manual inspection, but also, compared with other machine learning methods, the present application can more accurately identify the defect level of the insulator, thereby improving the operating safety and maintenance efficiency of the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A schematic diagram of a process of an intelligent detection method for insulator defects according to an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of the structure of a feature pyramid network of a gradient pyramid decomposition method according to an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram showing the fusion of an infrared image and a visible light image according to an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram showing a prediction result of a center core wire of an insulator according to an embodiment of the present application;

[0024] Figure 5 A schematic structural diagram of an intelligent detection device for insulator defects according to an embodiment of the present application is shown;

[0025] Figure 6 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0027] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0028] At present, the detection of insulator defects usually adopts visual inspection, electrical testing, ultrasonic testing, laser scanning, or single infrared thermal imaging technology, etc., but these technologies generally have the problems of high cost of detection equipment, need for special equipment and personnel, strict detection conditions, poor real-time performance, and low accuracy of single infrared thermal imaging. In view of the above problems, this application is specially proposed. The main idea of ​​this application is to fuse the infrared image and the visible light image data, and use a machine learning model to detect and identify insulator defects based on the fused data. Without strict conditions and professional equipment, effective identification of insulator defects can be achieved, and compared with the existing single infrared thermal imaging technology, it has higher accuracy.

[0029] Figure 1 A schematic diagram of a flow chart of an intelligent detection method for insulator defects according to an embodiment of the present application is shown. Figure 1 It can be seen that this embodiment includes steps S110 to S150:

[0030] Step S110, obtaining an infrared image of the insulator to be inspected, preprocessing the infrared image and extracting feature values ​​to obtain an infrared preprocessed image and infrared image feature values.

[0031] The application of infrared imaging detection technology in power systems is mainly to capture the thermal radiation on the surface of equipment through thermal imaging cameras, so as to detect and diagnose thermal anomalies of equipment, which are usually related to equipment failures or defects, such as poor contact, overheating, partial discharge, etc. Infrared imaging can provide intuitive images of the temperature distribution on the surface of equipment. By quantitatively analyzing these images, key parameters reflecting the status of the equipment can be extracted.

[0032] In some embodiments of the present application, a drone or a fixed camera can be used to collect infrared images of insulators of a transmission line. The drone or camera should be equipped with a corresponding infrared sensor to ensure that clear image data can be captured.

[0033] In order to extract quantitative parameters from the infrared imaging test results, the image needs to be preprocessed first, including image denoising, contrast enhancement and pseudo-color mapping, to improve the recognizability of temperature differences in the image. Then, the different areas of the device are separated by image segmentation technology, especially the hot spot area is distinguished from the normal area. On this basis, parameters such as the temperature value, temperature variation range, hot spot area and shape characteristics of the hot spot area can be calculated. The temperature value is the most direct quantitative parameter, which reflects the temperature distribution on the surface of the device. The temperature variation range can indicate the temperature gradient of the device in different parts, which is crucial for identifying problems such as local overheating or thermal runaway. The hot spot area and shape characteristics provide information about the size and shape of the fault area, which helps to determine the severity and possible causes of the fault. In addition, by analyzing the time series of a series of infrared images, the trend of the temperature change of the device over time can be monitored, so as to realize the dynamic monitoring of the device status. This trend analysis is of great significance for predicting the occurrence of potential equipment failures. In order to improve the quantitative analysis accuracy of infrared imaging test results, researchers will also consider the influence of environmental factors on the test results, such as ambient temperature, humidity, wind speed, etc., and take corresponding correction measures. Through these methods, infrared imaging detection technology can provide scientific and accurate data support for the maintenance and fault diagnosis of power systems.

[0034] In some embodiments of the present application, after the infrared image data information of the insulator to be detected is obtained, it is preprocessed. In some embodiments, preprocessing the infrared image data information includes: grayscale processing the infrared image to obtain an infrared grayscale image; binarization conversion processing the infrared grayscale image to obtain an infrared binary image; background area cutting processing of the infrared binary image to obtain an infrared preprocessed image containing a temperature abnormality area.

[0035] In one feasible embodiment, grayscale processing is performed on the collected infrared image, and grayscale processing is performed on each pixel in the image using Formula 1:

[0036] Y = 0.299R + 0.587G + 0.114B (1);

[0037] In formula 1, R, G, and B are the values ​​of the red, green, and blue color channels, respectively.

[0038] Then, the threshold can be automatically determined using, but not limited to, the OTSU method (Otsu's method, an automatic threshold selection technique for image processing) to convert the grayscale image into a binary image.

[0039] Finally, the background area is cut to remove the non-target area in the binary image, and only the temperature abnormality area is retained, which is recorded as the infrared preprocessed image.

[0040] Then, the characteristic value of the infrared image is extracted. As mentioned above, the thermal anomaly in the infrared image is usually related to the failure or defect of the equipment. Therefore, the location information of the defect can be preliminarily determined based on the temperature change in the infrared image.

[0041] Therefore, feature value extraction is performed on the infrared image, and the feature value mainly represents where the temperature rise occurs in the infrared image.

[0042] Specifically, in some embodiments, extracting the feature value of the infrared image includes: extracting the insulator disk area in the infrared grayscale image using the maximum inter-class variance method based on the infrared grayscale image generated in the preprocessing process to obtain the infrared image disk area; extracting the maximum temperature rise T of the infrared image disk area max As the infrared image characteristic value. That is, the temperature information is extracted from the infrared image, and the maximum temperature rise Tmax=max(T(x,y)) is calculated as the infrared image characteristic value.

[0043] Step S120 , obtaining a visible light image of the insulator to be inspected, preprocessing the visible light image and extracting feature values ​​to obtain a visible light preprocessing image and visible light image feature values.

[0044] In some embodiments of the present application, a drone or a fixed camera can be used to collect visible light images of insulators of a transmission line. The drone or camera should be equipped with a corresponding visible light sensor to ensure that clear image data can be captured.

[0045] After obtaining the visible light image of the insulator to be inspected, it is preprocessed, including but not limited to color temperature correction, exposure correction, image grayscale processing, enhancement and denoising, etc., and the grayscale image is further extracted from the insulator disk area using the seed region growing method. Specifically, in some embodiments of the present application, the visible light image is preprocessed to obtain a visible light preprocessed image, including: before obtaining the visible light image, sequentially performing color temperature correction on the camera using a white card and performing exposure correction on the camera using a gray card; after obtaining the visible light image, performing grayscale processing on the visible light image to obtain a visible light grayscale image; performing enhancement and denoising on the visible light grayscale image to obtain a visible light preprocessed image.

[0046] Specifically, the camera is first corrected for color temperature, such as using a white card to correct the camera for color temperature to eliminate the influence of the color temperature of the ambient light; then a gray card is used to correct the camera for exposure to eliminate the influence of the ambient light illumination. More specifically, the white card correction can be performed according to the following method: use the camera to be corrected to take a picture of a white card, the brightness value of the white card should be close to the brightest part of the scene, calculate the brightness value of the white card image, and compare it with the ideal brightness value, adjust the camera exposure setting according to the difference, so that the brightness value of the white card is close to the ideal value. The gray card correction can be performed according to the following method: use the camera to be corrected to take a picture of a gray card, the grayscale value of the gray card should be close to the average grayscale of the scene, calculate the grayscale value of the gray card image, and compare it with the ideal grayscale value, adjust the camera exposure setting according to the difference, so that the grayscale value of the gray card is close to the ideal value.

[0047] The corrected camera is then used to collect visible light images, which are then enhanced and denoised, such as using histogram equalization to enhance image contrast and using median filtering to remove image noise, in order to improve image quality and reduce environmental impact.

[0048] Then, the visible light image is extracted for feature values. In some embodiments, the feature value extraction for the visible light image includes: extracting the insulator disk area based on the visible light grayscale image generated in the preprocessing process using the seed region growth method to obtain the visible light image disk area; extracting the U component mean and the V component mean of the visible light image disk area in the YUV color space; comparing the absolute value of the difference between the U component mean of the color of the dirty insulator disk of level 0 and level IV, and the absolute value of the difference between the V component mean of the color of the dirty insulator disk of level 0 and level IV, and taking the component corresponding to the larger absolute value as the feature value of the visible light image. That is, extracting the mean of the U and V components of the YUV color space from the visible light image, and selecting the component with the larger difference as the feature value of the visible light image.

[0049] Specifically, the following formula can be used to convert the color space. For a pixel point, the pixel value in its grayscale image is Y, and the expression of Y is as shown in Formula 1. The conversion process is as shown in Formulas 2 to 4:

[0050] Y = 0.299R + 0.587G + 0.114B Formula (1);

[0051] U=BY,V=RY Formula (2);

[0052]

[0053]

[0054] Where U(x,y) and V(x,y) are the component values ​​of U and V in the YUV color space, R is the insulator disk area, and A is the area of ​​the area.

[0055] In contaminated insulators, the U component usually refers to a certain spectral characteristic of the color of the insulator disk under contaminated conditions. For Class 0 and Class IV contaminated insulators, the color of the U component is usually different. Class 0 contamination: relatively clean, usually lighter in color, the value of the U component may be higher, close to white or transparent, indicating that there is less dirt on the surface of the insulator. Class IV contamination: This level of contamination is heavier, the disk color may become darker gray or black, and the value of the U component is lower, suggesting that the dirt covers more and the insulation performance may be affected. The same is true for the V component.

[0056] After obtaining the U component and the V component, compare the absolute value of the difference between the mean values ​​of the U component of the color of the grade 0 and grade IV contaminated insulator disks and the absolute value of the difference between the mean values ​​of the V component of the color of the grade 0 and grade IV contaminated insulator disks, and take the component corresponding to the larger one as the visible light image feature value.

[0057] It should be noted that both the infrared image and the visible light image are images containing the insulator to be detected, and the framing of the two images can be the same or different, preferably the same; the sizes of the two images can be the same or different, preferably the same.

[0058] Step S130 : Based on the infrared image feature value and the visible light image feature value, a gradient pyramid decomposition algorithm is used to fuse the infrared pre-processed image and the visible light pre-processed image to obtain a fused image and location information of insulator defects.

[0059] The preprocessed infrared image and the visible light image are fused, and the position information of the insulator defects of the insulator can be obtained during the fusion process.

[0060] Specifically, in some embodiments, the infrared preprocessed image and the visible light preprocessed image are fused using a gradient pyramid decomposition algorithm based on the infrared image eigenvalues ​​and the visible light image eigenvalues ​​to obtain a fused image, including: determining the position information of the insulator defect in the infrared preprocessed image based on the infrared image eigenvalues, and preliminarily locating the insulator defect in the visible light preprocessed image according to the position information combined with the visible light eigenvalues; identifying feature points using a gradient pyramid decomposition method, and removing feature points with the same feature vectors using a grayscale statistics local feature matching method; matching feature points between the infrared preprocessed image and the visible light preprocessed image after preliminary positioning, and interpolating the visible light preprocessed image according to the matching results to process the visible light preprocessed image into an image corresponding to the position of the infrared preprocessed image; fusing the infrared preprocessed image and the visible light preprocessed image based on NSCT transformation, and outputting the position information of the insulator defect; sharpening and enhancing the fused image to obtain a fused image.

[0061] That is to say, the eigenvalues ​​extracted in the above steps can be used for preliminary positioning, and then the gradient pyramid decomposition method can be used for accurate fusion. In this process, the location information of the insulator defects can be obtained.

[0062] First, the position information of the insulator defect in the infrared preprocessed image is determined based on the infrared image characteristic value, and the insulator defect is preliminarily located in the visible light preprocessed image according to the position information combined with the visible light characteristic value. Generally speaking, the resolution of infrared images is relatively low, and the resolution of visible light images is higher and clearer. The position of the insulator defect can be roughly identified according to the infrared image characteristic value, and the insulator defect is located by the infrared image characteristic value, and the position information of the insulator defect in the infrared preprocessed image is determined. The position information is matched with the visible light characteristic value, so as to preliminarily locate the insulator defect in the visible light preprocessed image. The positioning result is a preliminary and relatively rough result.

[0063] Then, the gradient pyramid decomposition method is used for more refined fusion. The gradient pyramid decomposition method is a commonly used image fusion technology that can effectively combine the information of multiple images to create a more visually impactful result. Figure 2 A schematic diagram of the structure of a feature pyramid network of a gradient pyramid decomposition method according to an embodiment of the present application is shown. Figure 2It can be seen that the feature pyramid network used to extract feature points is in the shape of a pyramid as a whole. The network can enhance the key features in the image. It contains multiple layers of sub-networks. After processing, multiple feature points of the infrared pre-processed image and the visible light pre-processed image can be obtained respectively.

[0064] A local feature matching method based on grayscale statistics is used to remove feature points with the same feature vectors, so that subsequent feature point matching is more accurate.

[0065] After deduplication, feature point matching is performed on the infrared preprocessed image and the visible light preprocessed image, and interpolation calculation is performed on the visible light preprocessed image according to the matching result, so as to process the visible light preprocessed image into an image corresponding to the position of the infrared preprocessed image.

[0066] Finally, the infrared and visible light images are fused based on NSCT transform to improve the spatial and frequency resolution of the image. Figure 3 A schematic diagram showing the fusion of an infrared image and a visible light image according to an embodiment of the present application is shown. Figure 3 It can be seen that after NSCT transformation, the infrared image and the visible light image form a fused image. In the fused image, the location information of the insulator defect can be easily seen (the colored part in the figure). Furthermore, the fused image can be sharpened, enhanced and other post-processing operations to improve the visual effect and usability of the image. The above advanced preliminary fusion detects the fusion area of ​​the insulator string on this basis, and performs a second fusion on this basis, and saves the results, which can quickly locate the position of the insulator string.

[0067] In the process of fusing the features and the images, the extracted feature points may be weightedly fused to emphasize important features, such as weighted fusion of the extracted features, such as using Formula 5 to adjust the importance of the features:

[0068] F=αT max +β(U mean +V mean ) formula (5);

[0069] Among them, α and β are weight coefficients, and the size of α and β can be adjusted according to the importance of the feature.

[0070] In some embodiments, the above-mentioned nonlinear principal component analysis-based NSCT transform fuses the infrared pre-processed image and the visible light pre-processed image, including: using a frequency domain analysis method to perform wavelet transform on the infrared pre-processed image and the visible light pre-processed image, respectively, to obtain a first wavelet coefficient and a second wavelet coefficient, respectively; in the wavelet transform domain, comparing and selecting the first wavelet coefficient and the second wavelet coefficient layer by layer and scale by scale, and retaining the wavelet coefficient containing more defect information; and reconstructing the retained wavelet coefficients into a fused image by using an inverse wavelet transform.

[0071] Step S140, using a pre-trained insulator defect detection deep learning model to perform defect recognition on the fused image to obtain an insulator defect grade.

[0072] For the identification of insulator defect levels, the present application adopts an insulator defect detection deep learning model based on deep learning. Specifically, the insulator defect detection deep learning model is constructed and pre-trained according to the following method: constructing an insulator defect detection deep learning model, the insulator defect detection deep learning model includes: a pyramid network, the pyramid network includes multiple residual blocks and a pyramid pooling block, each of the residual blocks contains multiple convolutional layers and batch normalization layers, the pyramid pooling block includes multiple pooling layers of different sizes, each of the pooling layers is followed by a convolutional layer; the top of the pyramid network is also linked to at least one fully connected layer; constructing a training sample set, the training sample set at least includes a training set, the training set includes multiple one-to-one corresponding infrared images and visible light images, the infrared images and the visible light images are annotated with insulator defect levels; the training sample set is input into the insulator defect detection deep learning model, the image features of the training samples are extracted and fused through the pyramid network, and the fused diversified features are mapped to the preset defect level through the fully connected layer to obtain the insulator defect level.

[0073] Specifically, a large amount of infrared image data and visible light image data of insulators are collected and their defect levels are annotated as training data. The training data can be divided into two parts, one is a training set and the other is a validation set. Using a deep learning algorithm and a feature pyramid network, image features are learned and classified, and a machine learning model is trained to identify the defect level of the insulator. In this application, the pyramid network is usually composed of multiple residual blocks and a pyramid pooling module. Each residual block contains several convolutional layers and batch normalization layers for extracting features; the pyramid pooling module is used to capture contextual information at different scales.

[0074] Taking a pair of infrared images and visible light images as an example, they are input into the deep learning model of insulator defect detection. The pyramid network is used to extract and fuse the image features of the training samples. Then, one or more fully connected layers are used to map the fused features to the prediction of the defect level. The output layer uses the softmax function for multi-classification to obtain the defect level of the insulator.

[0075] The model is trained on the training set, the validation set is used to monitor the performance of the model, and the hyperparameters are adjusted to improve the accuracy of the model, thereby obtaining a deep learning model for insulator defect detection.

[0076] In the prediction process, the trained insulator defect detection deep learning model is used to analyze the fused image data, identify the defects of the insulator, classify and predict the quality of the insulator, and use the machine learning model to classify the defects according to the degree of deviation from the core wire and the characteristics of the abnormal area. Figure 4 , Figure 4 A schematic diagram of the prediction results of the center core of an insulator according to an embodiment of the present application is shown, and the defects of the insulator are classified and predicted, and the levels are divided according to the degree of deviation from the center core and the characteristics of the abnormal area.

[0077] In one embodiment, the present application designs a pyramid pooling module including 4 scale pooling layers, each scale pooling layer is followed by a 1x1 convolution layer for channel dimensionality reduction. The 4 scale pooling layers are: scale 1: 2×2 pooling; scale 2: 4×4 pooling; scale 3: 6×6 pooling; scale 4: 8×8 pooling.

[0078] During the training process, feature maps of different scales are obtained after extraction. These feature maps of different scales are upsampled and spliced ​​to form a fused feature map. Then, bilinear interpolation is used for upsampling, and feature maps of different scales are spliced ​​to increase the diversity of features and obtain a fused image.

[0079] At the top of the pyramid network, one or more fully connected layers are used to map the features of the fused image to the prediction of the defect level, and the output layer uses a softmax function for multi-classification. In some embodiments of the present application, the model can be trained using a cross entropy loss function, using an Adma optimizer with a learning rate set to 0.001. The model is trained on the training set and the validation set is used to monitor the performance of the model. Hyperparameters are adjusted to improve the accuracy of the model.

[0080] In the prediction process, a new image is input, and the trained model is used to predict the new image, and the probability distribution of each defect is output. The defect level is determined according to the highest probability, the defect location is marked on the image, and different colors are used according to the level to generate a label for each defect area to show its predicted level. In some embodiments of the present application, the setting and judgment of the defect level can be achieved as follows:

[0081]

[0082] Among them, θ1, θ2, θ3, θ4 are preset thresholds.

[0083] As for the output of the results, in some embodiments of the present application, the prediction results can be output in a visual manner, including marking of defect locations, indication of defect levels, etc., to provide decision support for the maintenance and inspection of the power system.

[0084] The following are some examples of specific application scenarios to better illustrate this application:

[0085] Example 1: Drone inspection operation process

[0086] Background: During a regular inspection of power lines, drones equipped with infrared cameras and visible light cameras were used to inspect the insulators of the transmission lines.

[0087] Steps:

[0088] 1. Image data acquisition:

[0089] The drone flew over the transmission line and simultaneously captured infrared and visible light images of the insulators.

[0090] 2. Image preprocessing:

[0091] The collected image is preprocessed based on Y=0.299R+0.587G+0.114B, where R, G, and B are the values ​​of the three color channels of red, green, and blue, respectively. The threshold is automatically determined using the OTSU method, the grayscale image is converted into a binary image, and then the background area is cut to remove the non-target area in the binary image, leaving only the temperature abnormality area.

[0092] Perform color temperature correction on visible light images and use a white card to correct the color temperature of the camera to eliminate the influence of the color temperature of ambient light. Use a gray card to correct the exposure of the camera to eliminate the influence of the illumination of ambient light.

[0093] Then grayscale processing, enhancement and denoising are performed.

[0094] 3. Feature extraction:

[0095] Extract temperature information from infrared images and calculate the maximum temperature rise T max =max(T(x,y)) as the infrared image feature value. Extract the mean of the U and V components of the YUV color space from the visible light image, and select the component with the largest difference as the feature value.

[0096] Conversion color space: Y = 0.299R + 0.587G + 0.114B, U = BY, V = RY

[0097]

[0098]

[0099] Among them, U(x,y) and V(x,y) are the component values ​​of U and V in the YUV color space, R is the insulator disk area, and A is the area of ​​the area.

[0100] 4. Image fusion and defect recognition:

[0101] The gradient pyramid decomposition algorithm is used to fuse infrared and visible light images to identify the fault location.

[0102] The machine learning model is used to identify insulator defects and classify them into different levels. The defect levels are as follows:

[0103]

[0104] θ1, θ2, θ3, θ4 are preset thresholds.

[0105] 5. Result output:

[0106] The identification results are output in a visual manner to guide the maintenance team in maintenance.

[0107] Result: The defect location and grade of the insulator were successfully identified, which improved the efficiency and accuracy of inspection.

[0108] Example 2: Laboratory simulation test

[0109] Background: In a laboratory setting, researchers need to validate the effectiveness of an assay.

[0110] Steps:

[0111] 1. Analog image acquisition:

[0112] Infrared and visible light images of insulator samples with artificial defects were taken under simulated high electric field environment.

[0113] 2. Image preprocessing and feature extraction:

[0114] The acquired images are subjected to the same preprocessing and feature extraction steps as in Example 1.

[0115] 3. Defect identification and classification:

[0116] Use machine learning models to identify defects in simulation data and classify them into grades.

[0117] Results: The laboratory test results were consistent with expectations, verifying the accuracy and reliability of the detection method.

[0118] Example 3: Software system development and application

[0119] Background: Development of a software system to automatically process infrared and visible light image data and identify insulator defects.

[0120] Steps:

[0121] 1. Software Development:

[0122] Write program code to implement functions such as image preprocessing, feature extraction, image fusion and defect recognition.

[0123] 2. Data input and processing:

[0124] Input infrared and visible light image data acquired in the field or in the laboratory into the software system.

[0125] 3. Automated testing:

[0126] The software automates the image analysis process and outputs defect locations and grades.

[0127] Results: The software system can accurately and quickly identify insulator defects, improving detection efficiency and accuracy.

[0128] Depend on Figure 1It can be seen from the method shown that the present application obtains the infrared image of the insulator to be detected, preprocesses and extracts the feature value of the infrared image, and obtains the infrared preprocessed image and the infrared image feature value; and obtains the visible light image of the insulator to be detected, preprocesses and extracts the feature value of the visible light image, and obtains the visible light preprocessed image and the visible light image feature value; then based on the infrared image feature value and the visible light image feature value, the infrared preprocessed image and the visible light preprocessed image are fused by using the gradient pyramid decomposition algorithm to obtain the fused image and the location information of the insulator defect; finally, the pre-trained insulator defect detection deep learning model is used to identify the defect of the fused image to obtain the insulator defect level. Compared with the prior art, the present invention uses infrared images and visible light images to intelligently identify the defect level of the insulator, which not only reduces the risk and cost of manual inspection, but also, compared with other machine learning methods, the present application can more accurately identify the defect level of the insulator, thereby improving the operation safety and maintenance efficiency of the power system.

[0129] Figure 5 A schematic diagram of the structure of an intelligent detection device for insulator defects according to an embodiment of the present application is shown. Figure 5 It can be seen that the intelligent detection device 500 for insulator defects includes:

[0130] The first acquisition unit 510 is used to acquire an infrared image of the insulator to be inspected, preprocess the infrared image and extract feature values ​​to obtain an infrared preprocessed image and infrared image feature values;

[0131] The second acquisition unit 520 is used to acquire a visible light image of the insulator to be inspected, preprocess the visible light image and extract feature values ​​to obtain a visible light preprocessed image and visible light image feature values;

[0132] A fusion unit 530 is used to fuse the infrared pre-processed image and the visible light pre-processed image by using a gradient pyramid decomposition algorithm based on the infrared image feature value and the visible light image feature value to obtain a fused image and location information of insulator defects;

[0133] The prediction unit 540 is used to use a pre-trained insulator defect detection deep learning model to perform defect recognition on the fused image to obtain an insulator defect grade.

[0134] In some embodiments of the present application, in the above-mentioned device, the first acquisition unit 510 is used to perform grayscale processing on the infrared image to obtain an infrared grayscale image; perform binarization conversion processing on the infrared grayscale image to obtain an infrared binary image; perform background area cutting processing on the infrared binary image to obtain an infrared pre-processed image containing a temperature abnormality area.

[0135] In some embodiments of the present application, in the above-mentioned device, the second acquisition unit 520 is used to, before acquiring the visible light image, sequentially perform color temperature correction on the camera using a white card and perform exposure correction on the camera using a gray card; after acquiring the visible light image, perform grayscale processing on the visible light image to obtain a visible light grayscale image; and perform enhancement and denoising processing on the visible light grayscale image to obtain a visible light preprocessed image.

[0136] In some embodiments of the present application, in the above-mentioned device, the first acquisition unit 510 is used to extract the insulator disk area in the infrared grayscale image generated in the preprocessing process by using the maximum inter-class variance method to obtain the infrared image disk area; extract the maximum temperature rise T of the infrared image disk area max As the infrared image feature value.

[0137] In some embodiments of the present application, in the above-mentioned device, the second acquisition unit 520 is used to extract feature values ​​of the visible light image according to the following method: based on the visible light grayscale image generated in the preprocessing process, the seed region growing method is used to extract the insulator disk area to obtain the visible light image disk area; the U component mean and the V component mean of the visible light image disk area in the YUV color space are extracted; the absolute value of the difference between the U component mean values ​​of the 0-level and IV-level contaminated insulator disk colors and the absolute value of the difference between the V component mean values ​​of the 0-level and IV-level contaminated insulator disk colors are compared, and the component corresponding to the larger absolute value is used as the visible light image feature value.

[0138] In some embodiments of the present application, in the above-mentioned device, the fusion unit 530 is used to determine the position information of the insulator defect in the infrared preprocessed image based on the infrared image eigenvalue, and preliminarily locate the insulator defect in the visible light preprocessed image according to the position information combined with the visible light image eigenvalue; use the gradient pyramid decomposition method to identify feature points, and use the grayscale statistics local feature matching method to remove feature points with the same feature vector; match the feature points of the infrared preprocessed image and the visible light preprocessed image after preliminary positioning, and interpolate the visible light preprocessed image according to the matching result to process the visible light preprocessed image into an image corresponding to the position of the infrared preprocessed image; fuse the infrared preprocessed image and the visible light preprocessed image based on the nonlinear principal component analysis NSCT transform, and output the position information of the insulator defect; sharpen and enhance the fused image to obtain a fused image.

[0139] In some embodiments of the present application, in the above-mentioned device, the fusion unit 530 is used to perform wavelet transform on the infrared pre-processed image and the visible light pre-processed image respectively using a frequency domain analysis method to obtain a first wavelet coefficient and a second wavelet coefficient respectively; in the wavelet transform domain, the first wavelet coefficient and the second wavelet coefficient are compared and selected layer by layer and scale by scale to retain the wavelet coefficients containing more defect information; and in the inverse wavelet transform, the retained wavelet coefficients are reconstructed into a fused image.

[0140] In some embodiments of the present application, in the above-mentioned device, the insulator defect detection deep learning model is trained according to the following method: constructing an insulator defect detection deep learning model, the insulator defect detection deep learning model includes: a pyramid network, the pyramid network includes multiple residual blocks and a pyramid pooling block, each of the residual blocks contains multiple convolutional layers and batch normalization layers, the pyramid pooling block includes multiple pooling layers of different sizes, and each of the pooling layers is followed by a convolutional layer; the top of the pyramid network is also linked to at least one fully connected layer; constructing a training sample set, the training sample set at least includes a training set, the training set includes multiple one-to-one corresponding infrared images and visible light images, and the infrared images and the visible light images are annotated with insulator defect levels; inputting the training sample set into the insulator defect detection deep learning model, extracting image features of the training samples through the pyramid network and fusing them, and mapping the fused diversified features to a preset defect level through the fully connected layer to obtain the insulator defect level.

[0141] It can be understood that the above-mentioned intelligent detection device for insulator defects can implement the various steps of the intelligent detection method for insulator defects provided in the aforementioned embodiments, and the relevant explanations on the intelligent detection method for insulator defects are applicable to the intelligent detection device for insulator defects, which will not be repeated here.

[0142] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 6 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0143] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0144] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0145] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an intelligent detection device for insulator defects at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0146] Acquire an infrared image of the insulator to be inspected, perform preprocessing and feature value extraction on the infrared image, and obtain an infrared preprocessing image and infrared image feature values;

[0147] Acquire a visible light image of the insulator to be inspected, perform preprocessing and feature value extraction on the visible light image, and obtain a visible light preprocessing image and visible light image feature values;

[0148] Based on the infrared image feature value and the visible light image feature value, a gradient pyramid decomposition algorithm is used to fuse the infrared pre-processed image and the visible light pre-processed image to obtain a fused image and location information of insulator defects;

[0149] A pre-trained insulator defect detection deep learning model is used to perform defect recognition on the fused image to obtain the insulator defect grade.

[0150] It should be noted that the above-mentioned intelligent detection device for insulator defects can implement the above-mentioned intelligent detection method for insulator defects one by one, and will not be described in detail.

[0151] The above application Figure 5The method performed by the intelligent detection device for insulator defects disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0152] The electronic device may also perform Figure 5 The method of implementing the intelligent detection device for insulator defects in the invention is realized by the intelligent detection device for insulator defects in Figure 5 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0153] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 5 The method performed by the intelligent detection device for insulator defects in the illustrated embodiment is specifically used to perform:

[0154] Acquire an infrared image of the insulator to be inspected, perform preprocessing and feature value extraction on the infrared image, and obtain an infrared preprocessing image and infrared image feature values;

[0155] Acquire a visible light image of the insulator to be inspected, perform preprocessing and feature value extraction on the visible light image, and obtain a visible light preprocessing image and visible light image feature values;

[0156] Based on the infrared image feature value and the visible light image feature value, a gradient pyramid decomposition algorithm is used to fuse the infrared pre-processed image and the visible light pre-processed image to obtain a fused image and location information of insulator defects;

[0157] A pre-trained insulator defect detection deep learning model is used to perform defect recognition on the fused image to obtain the insulator defect grade.

[0158] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0159] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0160] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0162] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0163] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0164] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0165] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0166] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0167] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An intelligent detection method for insulator defects, characterized in that: include: Acquire an infrared image of the insulator to be inspected, perform preprocessing and feature value extraction on the infrared image, and obtain an infrared preprocessing image and infrared image feature values; Acquire a visible light image of the insulator to be inspected, perform preprocessing and feature value extraction on the visible light image, and obtain a visible light preprocessing image and visible light image feature values; Based on the infrared image feature value and the visible light image feature value, a gradient pyramid decomposition algorithm is used to fuse the infrared pre-processed image and the visible light pre-processed image to obtain a fused image and location information of insulator defects; A pre-trained insulator defect detection deep learning model is used to perform defect recognition on the fused image to obtain the insulator defect grade.

2. The method according to claim 1, characterized in that Preprocessing the infrared image includes: Performing grayscale processing on the infrared image to obtain an infrared grayscale image; Performing binarization conversion processing on the infrared grayscale image to obtain an infrared binary image; The infrared binary image is subjected to background region cutting processing to obtain an infrared preprocessed image containing a temperature abnormality region.

3. The method according to claim 2, characterized in that The feature value extraction of the infrared image is performed according to the following method: Based on the infrared grayscale image generated in the preprocessing process, the maximum inter-class variance method is used to extract the insulator disk area in the infrared grayscale image to obtain the infrared image disk area; Extract the maximum temperature rise T of the infrared image disk area max As the infrared image feature value.

4. The method according to claim 1, characterized in that Preprocessing the visible light image includes: Before acquiring the visible light image, color temperature correction of the camera is performed using a white card, and exposure correction of the camera is performed using a gray card; After acquiring the visible light image, performing grayscale processing on the visible light image to obtain a visible light grayscale image; The visible light grayscale image is enhanced and denoised to obtain a visible light preprocessed image.

5. The method according to claim 4, characterized in that The feature value extraction of the visible light image is performed according to the following method: Based on the visible light grayscale image generated in the preprocessing process, the seed region growing method is used to extract the insulator disk area to obtain the visible light image disk area; Extract the mean value of the U component and the mean value of the V component of the visible light image disk area in the YUV color space; Compare the absolute value of the difference between the mean values ​​of the U component of the color of the contaminated insulator disks of level 0 and level IV, and the absolute value of the difference between the mean values ​​of the V component of the color of the contaminated insulator disks of level 0 and level IV, and take the component corresponding to the larger absolute value as the visible light image feature value.

6. The method according to claim 1, characterized in that The method of fusing the infrared preprocessed image and the visible light preprocessed image using a gradient pyramid decomposition algorithm based on the infrared image feature value and the visible light image feature value to obtain a fused image and location information of insulator defects includes: Determine the position information of the insulator defect in the infrared preprocessed image based on the infrared image characteristic value, and preliminarily locate the insulator defect in the visible light preprocessed image according to the position information combined with the visible light image characteristic value; The gradient pyramid decomposition method is used to identify feature points, and the local feature matching method of grayscale statistics is used to remove feature points with the same feature vectors. Performing feature point matching on the infrared pre-processed image and the visible light pre-processed image after preliminary positioning, and performing interpolation calculation on the visible light pre-processed image according to the matching result, so as to process the visible light pre-processed image into an image corresponding to the position of the infrared pre-processed image; Based on nonlinear principal component analysis (NSCT) transformation, the infrared pre-processed image and the visible light pre-processed image are fused, and the position information of the insulator defect is output; The fused image is sharpened and enhanced to obtain a fused image.

7. The method according to claim 6, characterized in that The step of fusing the infrared pre-processed image and the visible light pre-processed image based on nonlinear principal component analysis (NSCT) transformation includes: The infrared pre-processed image and the visible light pre-processed image are respectively subjected to wavelet transformation using the frequency domain analysis method to obtain the first wavelet coefficient and the second wavelet coefficient respectively; In the wavelet transform domain, the first wavelet coefficient and the second wavelet coefficient are compared and selected layer by layer and scale by scale, and the wavelet coefficient containing more defect information is retained; Using inverse wavelet transform, the retained wavelet coefficients are reconstructed into a fused image.

8. The method according to claim 1, characterized in that: The insulator defect detection deep learning model is trained according to the following method: Constructing an insulator defect detection deep learning model, the insulator defect detection deep learning model comprising: a pyramid network, the pyramid network comprising a plurality of residual blocks and a pyramid pooling block, each of the residual blocks comprising a plurality of convolutional layers and a batch normalization layer, the pyramid pooling block comprising a plurality of pooling layers of different sizes, each of the pooling layers being followed by a convolutional layer; the top of the pyramid network is also connected to at least one fully connected layer; Constructing a training sample set, the training sample set at least comprising a training set, the training set comprising a plurality of one-to-one corresponding infrared images and visible light images, the infrared images and the visible light images being annotated with insulator defect levels; The training sample set is input into the insulator defect detection deep learning model, the image features of the training samples are extracted and fused through the pyramid network, and the fused diversified features are mapped to the preset defect level through the fully connected layer to obtain the insulator defect level.

9. An intelligent detection device for insulator defects, characterized in that: The device comprises: A first acquisition unit is used to acquire an infrared image of the insulator to be detected, preprocess the infrared image and extract feature values ​​to obtain an infrared preprocessed image and infrared image feature values; A second acquisition unit is used to acquire a visible light image of the insulator to be inspected, preprocess the visible light image and extract feature values ​​to obtain a visible light preprocessed image and a visible light image feature value; a fusion unit, configured to fuse the infrared pre-processed image and the visible light pre-processed image by using a gradient pyramid decomposition algorithm based on the infrared image feature value and the visible light image feature value, so as to obtain a fused image and location information of insulator defects; The prediction unit is used to use a pre-trained insulator defect detection deep learning model to perform defect recognition on the fused image to obtain the insulator defect level.

10. A computer-readable storage medium storing one or more programs, characterized in that: When the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device executes the intelligent detection method for insulator defects as claimed in any one of claims 1 to 7.

Images