Joint detection method and device for insulator defects, electronic equipment and medium

Abstract

The invention relates to a joint detection method and device for insulator defects, electronic equipment and a medium, and the method comprises the steps: obtaining an ultraviolet image of a to-be-detected insulator, carrying out the preprocessing of the ultraviolet image, and obtaining an ultraviolet preprocessing image containing a discharge light spot region; obtaining a visible light image of a to-be-detected insulator, and performing preprocessing and feature value extraction on the visible light image to obtain a visible light preprocessed image and a visible light image feature value; based on the ultraviolet preprocessing image, constructing an ultraviolet discharge spectrum of the insulator, and determining a defect discharge grade according to the ultraviolet discharge spectrum; based on the ultraviolet discharge spectrum and the visible light image feature value, the ultraviolet preprocessed image and the visible light preprocessed image are fused, and a fused image and position information of the insulator defect are obtained; and performing defect identification on the fused image by adopting a pre-trained insulator defect detection deep learning model to obtain an insulator defect grade. According to the invention, the detection accuracy of the degraded insulator is improved.

Description

Technical Field

[0001] The present application relates to the technical field of power system detection and monitoring, and in particular to a combined detection method, device, electronic equipment and medium for insulator defects. Background Art

[0002] In the power system, the safe and stable operation of transmission lines is crucial to ensuring social and economic activities and the daily lives of residents. As a key component in the transmission line, the main function of insulators is to support the conductors and prevent the current from returning to the ground, ensuring the safety and reliability of power transmission. However, insulators will be affected by a variety of external factors during long-term operation, such as strong electric fields, ultraviolet radiation, temperature changes, humidity, and pollutant deposition. These factors may cause defects such as dirt, aging, cracks, and corrosion on the surface of the insulator, thereby affecting its insulation performance and increasing the risk of transmission line failure.

[0003] At present, the detection of insulator defects usually adopts visual inspection, electrical testing, ultrasonic testing, laser scanning, or single ultraviolet thermal imaging technology, but these technologies generally have problems such as high cost of detection equipment, need for special equipment and personnel, strict detection conditions, poor real-time performance, and low accuracy of single ultraviolet thermal imaging. Summary of the invention

[0004] In view of the above problems, the embodiments of the present application provide a joint detection method, device, electronic device and medium for insulator defects to overcome or partially overcome the shortcomings of the prior art.

[0005] In a first aspect, the present application provides a method for detecting insulator defects, comprising:

[0006] Acquire an ultraviolet image of the insulator to be inspected, and preprocess the ultraviolet image to obtain an ultraviolet preprocessed image containing a discharge spot area;

[0007] Acquire a visible light image of the insulator to be inspected, perform preprocessing and feature value extraction on the visible light image, and obtain a visible light preprocessing image and visible light image feature values;

[0008] Based on the ultraviolet preprocessed image, constructing an ultraviolet discharge spectrum of the insulator, and determining a defect discharge level according to the ultraviolet discharge spectrum;

[0009] Based on the ultraviolet discharge spectrum and the visible light image characteristic value, the ultraviolet pre-processed image and the visible light pre-processed image are fused to obtain a fused image and location information of insulator defects;

[0010] A pre-trained insulator defect detection deep learning model is used to perform defect recognition on the fused image to obtain the insulator defect grade.

[0011] In a second aspect, the present application provides a combined detection device for insulator defects, the device comprising:

[0012] A first acquisition unit is used to acquire an ultraviolet image of the insulator to be inspected, and preprocess the ultraviolet image to obtain an ultraviolet preprocessed image containing a discharge spot area;

[0013] A second acquisition unit is used to acquire a visible light image of the insulator to be inspected, preprocess the visible light image and extract feature values ​​to obtain a visible light preprocessed image and a visible light image feature value;

[0014] A spectrum construction unit, used to construct an ultraviolet discharge spectrum of the insulator based on the ultraviolet preprocessed image, and determine the defect discharge level according to the ultraviolet discharge spectrum;

[0015] A fusion unit, configured to fuse the ultraviolet preprocessed image and the visible light preprocessed image based on the ultraviolet discharge spectrum and the visible light image characteristic value, so as to obtain a fused image and location information of insulator defects;

[0016] The prediction unit is used to use a pre-trained insulator defect detection deep learning model to perform defect recognition on the fused image to obtain the insulator defect level.

[0017] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes any of the above-mentioned joint detection methods for insulator defects.

[0018] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs, the electronic device performs any of the above-mentioned joint detection methods for insulator defects.

[0019] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0020] The present application obtains an ultraviolet image of the insulator to be detected, preprocesses the ultraviolet image, and obtains an ultraviolet preprocessed image containing a discharge spot area; obtains a visible light image of the insulator to be detected, preprocesses the visible light image and extracts feature values ​​to obtain a visible light preprocessed image and a visible light image feature value; then, based on the ultraviolet preprocessed image, constructs an ultraviolet discharge spectrum of the insulator, and determines the defect discharge level according to the ultraviolet discharge spectrum; based on the ultraviolet discharge spectrum and the visible light image feature value, the ultraviolet preprocessed image and the visible light preprocessed image are fused to obtain a fused image and the location information of the insulator defect; finally, a pre-trained insulator defect detection deep learning model is used to identify defects on the fused image to obtain the insulator defect level. The defect discharge level and the insulator defect level together constitute the detection result. Compared with the prior art, the present invention uses ultraviolet images and visible light images to jointly identify the defect levels of insulators. By analyzing and studying the discharge characteristics of the equipment, the accuracy of detecting insulator defects is improved, the effectiveness of insulator detection is greatly improved, and the limitations of the use of a single device are reduced. It not only improves the detection accuracy of deteriorated insulators, but also has important significance in saving labor, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 A schematic flow chart of a joint detection method for insulator defects according to an embodiment of the present application is shown;

[0023] Figure 2 shows a collected ultraviolet image according to an embodiment of the present application;

[0024] Figure 3 An image after converting an ultraviolet image into a binary image according to an embodiment of the present application is shown;

[0025] Figure 4 A schematic diagram of the structure of a feature pyramid network of a gradient pyramid decomposition method according to an embodiment of the present application is shown;

[0026] Figure 5 A schematic structural diagram of a combined detection device for insulator defects according to an embodiment of the present application is shown;

[0027] Figure 6 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0029] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0030] At present, the detection of insulator defects usually adopts visual inspection, electrical testing, ultrasonic testing, laser scanning, or single ultraviolet thermal imaging technology, etc., but these technologies generally have the problems of high cost of detection equipment, need for special equipment and personnel, strict detection conditions, poor real-time performance, and low accuracy of single ultraviolet thermal imaging. In view of the above problems, this application is specially proposed. The main idea of ​​this application is to fuse the ultraviolet image and the visible light image data, and use a machine learning model to detect and identify insulator defects based on the fused data. Without strict conditions and professional equipment, effective identification of insulator defects can be achieved, and compared with the existing single ultraviolet thermal imaging technology, it has higher accuracy.

[0031] Figure 1 A schematic diagram of a process flow of a joint detection method for insulator defects according to an embodiment of the present application is shown. Figure 1 It can be seen that this embodiment includes steps S110 to S150:

[0032] Step S110, acquiring an ultraviolet image of the insulator to be inspected, and preprocessing the ultraviolet image to obtain an ultraviolet preprocessed image including a discharge spot area.

[0033] The application of UV imaging detection technology in power systems is mainly to visualize the discharge phenomenon of high-voltage electrical equipment. Through UV imaging, the UV light generated during discharge can be captured, thus forming visible light spots on the image. The size, brightness, and distribution of these light spots are important bases for evaluating the discharge intensity and equipment status.

[0034] In actual scenarios, before starting, check whether the meteorological environment meets the industry standards. If not, end the test. The industry standards based on some embodiments of this application are the meteorological environment requirements of "DLT 664-2008 Application Specifications for Ultraviolet Diagnosis of Energized Equipment" and "DLT345-2010 Application Guidelines for Ultraviolet Diagnosis Technology of Energized Equipment".

[0035] In some embodiments of the present application, a drone or a fixed camera can be used to collect ultraviolet images of insulators of transmission lines. The drone or camera should be equipped with a corresponding ultraviolet sensor to ensure that clear image data can be captured, such as Figure 2 As shown, Figure 2 The figure shows a collected ultraviolet image according to an embodiment of the present application.

[0036] After the ultraviolet image is acquired, it is preprocessed, including image grayscale processing, binary image conversion processing and background area cutting processing, to obtain an ultraviolet image of the discharge spot area.

[0037] In some embodiments of the present application, after obtaining the ultraviolet image data information of the insulator to be detected, it is preprocessed. In some embodiments, the preprocessing of the ultraviolet image data information includes: grayscale processing of the ultraviolet image to obtain an ultraviolet grayscale image; binarization conversion processing of the ultraviolet grayscale image to obtain an ultraviolet binary image; background area cutting processing of the ultraviolet binary image to obtain an image containing a discharge spot area; using a threshold segmentation or edge detection algorithm to determine the boundary of the discharge spot to obtain an ultraviolet preprocessed image.

[0038] In one feasible embodiment, grayscale processing is performed on the collected ultraviolet image, and grayscale processing is performed on each pixel in the image using Formula 1:

[0039] Y = 0.299R + 0.587G + 0.114B Formula (1);

[0040] In formula 1, R, G, and B are the values ​​of the red, green, and blue color channels, respectively.

[0041] Then, the threshold can be automatically determined using, but not limited to, the OTSU method (Otsu's method, an automatic threshold selection technique for image processing) to convert the grayscale image into a binary image, as shown in Formula 2. The result after conversion is Figure 3 shown.

[0042]

[0043] Among them, G(x,y) is the grayscale value of the original grayscale image at (x,y), and B(x,y) is the pixel value of the binarized image at (x,y).

[0044] Then the background area is cut to remove the non-target area in the binary image, and only the temperature abnormal area is retained to obtain an image containing the discharge spot area.

[0045] Finally, the boundary of the discharge spot is determined by using threshold segmentation or edge detection algorithm to obtain the UV preprocessed image.

[0046] The discharge spot area is identified in the image containing the discharge spot area. The image collected by the UV imager is a video stream. In order to quantitatively calculate the spot area, it mainly includes three parts: background difference, noise suppression and feature extraction. The principal component analysis method is suitable for background image extraction, electrical equipment is a still image, and the discharge spot is a moving image.

[0047] Specifically, a threshold segmentation or edge detection algorithm may be used to determine the boundary and area of ​​the discharge spot. The specific process is briefly described as follows.

[0048] First, the frame difference method is calculated based on the background difference principle, as shown in Formula 3:

[0049] ΔB t (x, y)=|B(x, y, t)-B(x, y, t-1)| Formula (3);

[0050] In formula 3, B(x, y, t) represents the gray value of the pixel at time t; △B t (x,y) represents the grayscale difference at time t.

[0051] Second, perform threshold judgment, as shown in Formula 4:

[0052]

[0053] In formula 4, ε is the threshold of background difference.

[0054] Third, noise suppression is mainly based on median filtering, threshold segmentation and morphological methods to achieve discharge spot contour extraction. Morphological processing mainly includes opening and closing operations, namely corrosion and expansion operations, as shown in Equation 5 and Equation 6 respectively:

[0055] AΘB={x|(B+x)∈A} Formula (5);

[0056]

[0057] Finally, the number of pixels in all boundary areas is counted to calculate the spot area. If the point with a gray value of 255 in the discharge area is defined as 1, the spot area is the total number of 1s in area A, S, as shown in Formula 7:

[0058] S=∑ (x，y)∈A 1 Formula (7).

[0059] Then, the characteristic values ​​of the ultraviolet image are extracted to construct the ultraviolet discharge spectrum of the insulator. Specifically, in some embodiments of the present application, the ultraviolet discharge spectrum of the insulator is constructed based on the ultraviolet pre-processed image, including: based on the ultraviolet pre-processed image, performing discharge analysis on the discharge spot area to extract discharge spectrum information related to the discharge activity; analyzing the spectral features in the discharge spectrum information, selecting representative characteristic values ​​of the discharge state of the insulator to construct the ultraviolet discharge spectrum of the insulator.

[0060] The ultraviolet preprocessed image is analyzed to extract the spectral information related to the discharge activity, and the spectral characteristics generated by the discharge are analyzed. The characteristic values ​​that can represent the discharge state of the insulator are selected to form the ultraviolet discharge spectrum of the insulator.

[0061] Step S120 , obtaining a visible light image of the insulator to be inspected, preprocessing the visible light image and extracting feature values ​​to obtain a visible light preprocessing image and visible light image feature values.

[0062] In some embodiments of the present application, a drone or a fixed camera can be used to collect visible light images of insulators of a transmission line. The drone or camera should be equipped with a corresponding visible light sensor to ensure that clear image data can be captured.

[0063] After obtaining the visible light image of the insulator to be inspected, it is preprocessed, including but not limited to color temperature correction, exposure correction, image grayscale processing, enhancement and denoising, etc., and the grayscale image is further extracted from the insulator disk area using the seed region growing method. Specifically, in some embodiments of the present application, the visible light image is preprocessed to obtain a visible light preprocessed image, including: before obtaining the visible light image, color temperature correction of the camera is performed using a white card, and exposure correction of the camera is performed using a gray card; after obtaining the visible light image, grayscale processing is performed on the visible light image to obtain a visible light grayscale image; the visible light grayscale image is enhanced and denoised; the insulator disk area is extracted using the seed region growing method to obtain a visible light preprocessed image containing the disk area.

[0064] Specifically, the camera is first corrected for color temperature, such as using a white card to correct the camera for color temperature to eliminate the influence of the color temperature of the ambient light; then a gray card is used to correct the camera for exposure to eliminate the influence of the ambient light illumination. More specifically, the white card correction can be performed according to the following method: use the camera to be corrected to take a picture of a white card, the brightness value of the white card should be close to the brightest part of the scene, calculate the brightness value of the white card image, and compare it with the ideal brightness value, adjust the camera exposure setting according to the difference, so that the brightness value of the white card is close to the ideal value. The gray card correction can be performed according to the following method: use the camera to be corrected to take a picture of a gray card, the grayscale value of the gray card should be close to the average grayscale of the scene, calculate the grayscale value of the gray card image, and compare it with the ideal grayscale value, adjust the camera exposure setting according to the difference, so that the grayscale value of the gray card is close to the ideal value.

[0065] The corrected camera is then used to collect visible light images, which are then enhanced and denoised, such as using histogram equalization to enhance image contrast and using median filtering to remove image noise, in order to improve image quality and reduce environmental impact.

[0066] Then, the disk area of ​​the insulator is extracted from the image to prepare for subsequent feature extraction and analysis. Specifically, the seed region growing method can be used to extract the disk area of ​​the insulator to obtain an image containing the disk area, which is recorded as a visible light preprocessing image.

[0067] According to the specific, select an initial seed point, which is located in the expected insulator disk area. For each pixel in the seed area, check its 8 neighboring pixels, and add the pixels adjacent to the seed point and meeting the conditions to the seed area according to the gray value. For each neighboring pixel, check whether it meets the growth criterion. If the neighboring pixel meets the growth criterion, add it to the seed area list.

[0068] Repeat the above process until no new pixels meet the conditions, where the gray value criterion is as shown in Formula 8:

[0069] |I(p)-I(q)|≤T Formula (8);

[0070] Among them, is the gray value of the I(p) seed point, is the gray value of the I(q) neighborhood pixel, and T is the preset threshold.

[0071] Then, feature value extraction is performed on the visible light preprocessed image. In some embodiments, feature value extraction on the visible light preprocessed image includes: extracting the mean value of the U component and the mean value of the V component of the disk area in the visible light preprocessed image in the YUV color space; comparing the absolute value of the difference between the mean value of the U component of the disk color of the class 0 and class IV contaminated insulators, and the absolute value of the difference between the mean value of the V component of the disk color of the class 0 and class IV contaminated insulators, and taking the component corresponding to the larger absolute value as the feature value of the visible light image. That is, extracting the mean value of the U and V components of the YUV color space from the visible light image, and selecting the component with the larger difference as the feature value of the visible light image.

[0072] Specifically, the following formula can be used to convert the color space. For a pixel point, the pixel value in its grayscale image is Y, and the expression of Y is as shown in Formula 9. The conversion process is as shown in Formulas 10 to 12:

[0073] Y = 0.299R + 0.587G + 0.114B Formula (9);

[0074] U=BY,V=RY Formula (10);

[0075]

[0076]

[0077] Among them, U(x,y) and V(x,y) are the component values ​​of U and V in the YUV color space, R is the insulator disk area, and A is the area of ​​the area.

[0078] In contaminated insulators, the U component usually refers to a certain spectral characteristic of the color of the insulator disk under contaminated conditions. For Class 0 and Class IV contaminated insulators, the color of the U component is usually different. Class 0 contamination: relatively clean, usually lighter in color, the value of the U component may be higher, close to white or transparent, indicating that there is less dirt on the surface of the insulator. Class IV contamination: This level of contamination is heavier, the disk color may become darker gray or black, and the value of the U component is lower, suggesting that the dirt covers more and the insulation performance may be affected. The same is true for the V component.

[0079] After obtaining the U component and the V component, compare the absolute value of the difference between the mean values ​​of the U component of the color of the grade 0 and grade IV contaminated insulator disks and the absolute value of the difference between the mean values ​​of the V component of the color of the grade 0 and grade IV contaminated insulator disks, and take the component corresponding to the larger one as the visible light image feature value.

[0080] It should be noted that both the ultraviolet image and the visible light image are images containing the insulator to be detected, and the framing of the two images can be the same or different, preferably the same; the sizes of the two images can be the same or different, preferably the same.

[0081] Step S130, constructing an insulator ultraviolet discharge spectrum based on the ultraviolet preprocessed image, and determining a defect discharge level.

[0082] According to the processing results of ultraviolet discharge spectrum, the severity of defective discharge is determined, and fuzzy logic and other methods are used to make fuzzy judgment on the discharge time of insulators to determine their defective discharge level.

[0083] Specifically, in some embodiments, determining the defective discharge level according to the ultraviolet discharge spectrum includes: defining a fuzzy set and a membership function to convert precise discharge time data into fuzzy values; and using a fuzzy logic inference algorithm to infer the defective discharge level based on the fuzzy values.

[0084] Define fuzzy sets and membership functions, convert precise discharge time data into fuzzy values, and then use the fuzzy inference system for judgment. Specifically, as shown in Formula 13:

[0085] Defect level = f(discharge time) formula (13);

[0086] Where f is the fuzzy logic function.

[0087] The membership function is shown in formula 14:

[0088]

[0089] Where t1, t2 are the thresholds of the discharge time.

[0090] Step S140, based on the ultraviolet discharge spectrum and the visible light image characteristic value, the ultraviolet pre-processed image and the visible light pre-processed image are fused to obtain a fused image and location information of insulator defects;

[0091] The preprocessed ultraviolet image and the visible light image are fused, and the position information of the insulator defects of the insulator can be obtained during the fusion process.

[0092] Specifically, in some embodiments, the ultraviolet preprocessed image and the visible light preprocessed image are fused based on the ultraviolet discharge spectrum and the characteristic values ​​of the visible light image to obtain a fused image, including: determining the position information of the insulator defect in the ultraviolet preprocessed image based on the characteristic values ​​in the ultraviolet discharge spectrum, and preliminarily locating the insulator defect in the visible light preprocessed image according to the position information combined with the visible light characteristic values; using a gradient pyramid decomposition method to identify feature points, and using a local feature matching method of grayscale statistics to remove feature points with the same feature vector; matching the feature points of the ultraviolet preprocessed image and the visible light preprocessed image after preliminary positioning, and interpolating the visible light preprocessed image according to the matching results to process the visible light preprocessed image into an image corresponding to the position of the ultraviolet preprocessed image; fusing the ultraviolet preprocessed image and the visible light preprocessed image based on NSCT transformation, and outputting the position information of the insulator defect; sharpening and enhancing the fused image to obtain a fused image.

[0093] That is to say, the eigenvalues ​​extracted in the above steps can be used for preliminary positioning, and then the gradient pyramid decomposition method can be used for accurate fusion. In this process, the location information of the insulator defects can be obtained.

[0094] First, the position information of the insulator defect in the ultraviolet preprocessed image is determined based on the ultraviolet image characteristic value, and the insulator defect is preliminarily located in the visible light preprocessed image according to the position information combined with the visible light characteristic value. Generally speaking, the resolution of ultraviolet images is relatively low, and the resolution of visible light images is higher and clearer. The position of the insulator defect can be roughly identified according to the ultraviolet image characteristic value, and the insulator defect is located by the ultraviolet image characteristic value, and the position information of the insulator defect in the ultraviolet preprocessed image is determined, and the position information is matched with the visible light characteristic value, so as to preliminarily locate the insulator defect in the visible light preprocessed image. The positioning result is a preliminary and relatively rough result.

[0095] Then, the gradient pyramid decomposition method is used for more refined fusion. The gradient pyramid decomposition method is a commonly used image fusion technology that can effectively combine the information of multiple images to create a more visually impactful result. Figure 4 A schematic diagram of the structure of a feature pyramid network of a gradient pyramid decomposition method according to an embodiment of the present application is shown. Figure 4It can be seen that the feature pyramid network used to extract feature points is in the shape of a pyramid as a whole. The network can enhance the key features in the image. It contains multiple layers of sub-networks. After processing, multiple feature points of the ultraviolet preprocessed image and the visible light preprocessed image can be obtained respectively.

[0096] A local feature matching method based on grayscale statistics is used to remove feature points with the same feature vectors, so that subsequent feature point matching is more accurate.

[0097] After deduplication, feature point matching is performed on the ultraviolet preprocessed image and the visible light preprocessed image, and interpolation calculation is performed on the visible light preprocessed image according to the matching result, so as to process the visible light preprocessed image into an image corresponding to the position of the ultraviolet preprocessed image.

[0098] Finally, the ultraviolet and visible light images are fused based on the NSCT transform to improve the spatial and frequency resolution of the image. Furthermore, the fused image can be sharpened, enhanced and other post-processing operations to improve the visual effect and usability of the image. The above advanced preliminary fusion detects the insulator string fusion area on this basis, and performs a second fusion on this basis, and saves the results, which can quickly locate the position of the insulator string.

[0099] Image fusion can be expressed by equation 15:

[0100] F=α*I 紫外 (x, y)+(1-α)*I 可见光 (x, y) Formula (15);

[0101] Among them, α is the weight coefficient.

[0102] In some embodiments, the above-mentioned nonlinear principal component analysis-based NSCT transform fuses the ultraviolet preprocessed image and the visible light preprocessed image, including: using a frequency domain analysis method to perform wavelet transform on the ultraviolet preprocessed image and the visible light preprocessed image, respectively, to obtain a first wavelet coefficient and a second wavelet coefficient, respectively; in the wavelet transform domain, comparing and selecting the first wavelet coefficient and the second wavelet coefficient layer by layer and scale by scale, and retaining the wavelet coefficient containing more defect information; and reconstructing the retained wavelet coefficients into a fused image by using an inverse wavelet transform.

[0103] Step S150, using a pre-trained insulator defect detection deep learning model to perform defect recognition on the fused image to obtain an insulator defect grade.

[0104] For the identification of insulator defect levels, the present application adopts an insulator defect detection deep learning model based on deep learning. Specifically, the insulator defect detection deep learning model is constructed and pre-trained according to the following method: constructing an insulator defect detection deep learning model, the insulator defect detection deep learning model includes: a pyramid network, the pyramid network includes multiple residual blocks and a pyramid pooling block, each of the residual blocks contains multiple convolutional layers and batch normalization layers, the pyramid pooling block includes multiple pooling layers of different sizes, each of the pooling layers is followed by a convolutional layer; the top of the pyramid network is also linked to at least one fully connected layer; constructing a training sample set, the training sample set at least includes a training set, the training set includes multiple one-to-one corresponding ultraviolet images and visible light images, the ultraviolet images and the visible light images are annotated with insulator defect levels; the training sample set is input into the insulator defect detection deep learning model, the image features of the training samples are extracted and fused through the pyramid network, and the fused diversified features are mapped to the preset defect level through the fully connected layer to obtain the insulator defect level.

[0105] Specifically, a large amount of ultraviolet image data and visible light image data of insulators are collected and their defect levels are annotated as training data. The training data can be divided into two parts, one is a training set and the other is a validation set. Using a deep learning algorithm and a feature pyramid network, image features are learned and classified, and a machine learning model is trained to identify the defect level of the insulator. In this application, the pyramid network is usually composed of multiple residual blocks and a pyramid pooling module. Each residual block contains several convolutional layers and batch normalization layers for extracting features; the pyramid pooling module is used to capture contextual information at different scales.

[0106] Taking a pair of ultraviolet images and visible light images as an example, they are input into the deep learning model of insulator defect detection. The pyramid network is used to extract and fuse the image features of the training samples, and then one or more fully connected layers are used to map the fused features to the prediction of the defect level. The output layer uses the softmax function for multi-classification to obtain the defect level of the insulator.

[0107] The model is trained on the training set, the validation set is used to monitor the performance of the model, and the hyperparameters are adjusted to improve the accuracy of the model, thereby obtaining a deep learning model for insulator defect detection.

[0108] During the prediction process, the trained insulator defect detection deep learning model is used to analyze the fused image data, identify insulator defects, classify and predict the quality of the insulator, and use a machine learning model to grade the defects based on the degree of deviation from the center core wire and the characteristics of the abnormal area.

[0109] In one embodiment, the present application designs a pyramid pooling module including 4 scale pooling layers, each scale pooling layer is followed by a 1x1 convolution layer for channel dimensionality reduction. The 4 scale pooling layers are: scale 1: 2×2 pooling; scale 2: 4×4 pooling; scale 3: 6×6 pooling; scale 4: 8×8 pooling.

[0110] During the training process, feature maps of different scales are obtained after extraction. These feature maps of different scales are upsampled and spliced ​​to form a fused feature map. Then, bilinear interpolation is used for upsampling, and feature maps of different scales are spliced ​​to increase the diversity of features and obtain a fused image.

[0111] At the top of the pyramid network, one or more fully connected layers are used to map the features of the fused image to the prediction of the defect level, and the output layer uses a softmax function for multi-classification. In some embodiments of the present application, the model can be trained using a cross entropy loss function, using an Adma optimizer with a learning rate set to 0.001. The model is trained on the training set and the validation set is used to monitor the performance of the model. Hyperparameters are adjusted to improve the accuracy of the model.

[0112] In the prediction process, a new image is input, and the trained model is used to predict the new image, and the probability distribution of each defect is output. The defect level is determined according to the highest probability, the defect location is marked on the image, and different colors are used according to the level to generate a label for each defect area to show its predicted level. In some embodiments of the present application, the setting and judgment of the defect level can be achieved as follows:

[0113]

[0114] Among them, θ1, θ2, θ3, θ4 are preset thresholds.

[0115] As for the output of the results, in some embodiments of the present application, the prediction results can be output in a visual manner, including marking of defect locations, indication of defect levels, etc., to provide decision support for the maintenance and inspection of the power system.

[0116] The final detection result can be formed based on the aforementioned defect discharge level and the defect level result predicted by the machine learning model. The two together provide decision support for the maintenance and inspection of the power system.

[0117] The following are some examples of specific application scenarios to better illustrate this application:

[0118] Depend on Figure 1It can be seen from the method shown that the present application obtains an ultraviolet image of the insulator to be detected, preprocesses the ultraviolet image, and obtains an ultraviolet preprocessed image containing a discharge spot area; obtains a visible light image of the insulator to be detected, preprocesses the visible light image and extracts feature values ​​to obtain a visible light preprocessed image and a visible light image feature value; then, based on the ultraviolet preprocessed image, constructs an ultraviolet discharge spectrum of the insulator, and determines the defect discharge level according to the ultraviolet discharge spectrum; based on the ultraviolet discharge spectrum and the visible light image feature value, the ultraviolet preprocessed image and the visible light preprocessed image are fused to obtain a fused image and the location information of the insulator defect; finally, a pre-trained insulator defect detection deep learning model is used to identify defects on the fused image to obtain the insulator defect level. The defect discharge level and the insulator defect level together constitute the detection result. Compared with the prior art, the present invention uses ultraviolet images and visible light images to jointly identify the defect levels of insulators. By analyzing and studying the discharge characteristics of the equipment, the accuracy of detecting insulator defects is improved, the effectiveness of insulator detection is greatly improved, and the limitations of the use of a single device are reduced. It not only improves the detection accuracy of deteriorated insulators, but also has important significance in saving labor, etc.

[0119] Figure 5 A schematic diagram of the structure of a combined detection device for insulator defects according to an embodiment of the present application is shown. Figure 5 It can be seen that the combined detection device 500 for insulator defects includes:

[0120] A first acquisition unit 510 is used to acquire an ultraviolet image of the insulator to be inspected, and preprocess the ultraviolet image to obtain an ultraviolet preprocessed image containing a discharge spot area;

[0121] The second acquisition unit 520 is used to acquire a visible light image of the insulator to be inspected, preprocess the visible light image and extract feature values ​​to obtain a visible light preprocessed image and visible light image feature values;

[0122] A spectrum construction unit 530, configured to construct an ultraviolet discharge spectrum of the insulator based on the ultraviolet preprocessed image, and determine a defect discharge level according to the ultraviolet discharge spectrum;

[0123] A fusion unit 540 is used to fuse the ultraviolet preprocessed image and the visible light preprocessed image based on the ultraviolet discharge spectrum and the visible light image characteristic value to obtain a fused image and location information of insulator defects;

[0124] The prediction unit 550 is used to use a pre-trained insulator defect detection deep learning model to perform defect recognition on the fused image to obtain an insulator defect level.

[0125] In some embodiments of the present application, in the above-mentioned device, the first acquisition unit 510 is used to perform grayscale processing on the ultraviolet image to obtain an ultraviolet grayscale image; perform binarization conversion processing on the ultraviolet grayscale image to obtain an ultraviolet binary image; perform background area cutting processing on the ultraviolet binary image to obtain an image containing the discharge spot area; use threshold segmentation or edge detection algorithm to determine the boundary of the discharge spot to obtain an ultraviolet preprocessed image.

[0126] In some embodiments of the present application, in the above-mentioned device, the spectrum construction unit 530 is used to perform discharge analysis on the discharge spot area based on the ultraviolet pre-processed image, and extract discharge spectrum information related to the discharge activity; analyze the spectral features in the discharge spectrum information, and select representative characteristic values ​​of the insulator discharge state to construct the ultraviolet discharge spectrum of the insulator.

[0127] In some embodiments of the present application, in the above-mentioned device, the second acquisition unit 520 is used to, before acquiring the visible light image, sequentially perform color temperature correction on the camera using a white card and perform exposure correction on the camera using a gray card; after acquiring the visible light image, perform grayscale processing on the visible light image to obtain a visible light grayscale image; perform enhancement and denoising on the visible light grayscale image; and use a seed region growing method to extract the insulator disk surface area to obtain a visible light preprocessed image containing the disk surface area.

[0128] In some embodiments of the present application, in the above-mentioned device, the second acquisition unit 520 is used to extract the U component mean and the V component mean of the disk area in the visible light preprocessed image in the YUV color space; compare the absolute value of the difference between the U component mean values ​​of the disk surface colors of grade 0 and grade IV contaminated insulators, and the absolute value of the difference between the V component mean values ​​of the disk surface colors of grade 0 and grade IV contaminated insulators, and take the component corresponding to the larger absolute value as the visible light image feature value.

[0129] In some embodiments of the present application, in the above-mentioned device, the spectrum construction unit 530 is used to define fuzzy sets and membership functions to convert precise discharge time data into fuzzy values; and adopt a fuzzy logic inference algorithm to infer the defective discharge level based on the fuzzy values.

[0130] In some embodiments of the present application, in the above-mentioned device, the fusion unit 540 is used to determine the position information of the insulator defect in the ultraviolet preprocessed image based on the eigenvalue in the ultraviolet discharge spectrum, and preliminarily locate the insulator defect in the visible light preprocessed image according to the position information combined with the eigenvalue of the visible light image; use the gradient pyramid decomposition method to identify feature points, and use the local feature matching method of grayscale statistics to remove feature points with the same eigenvectors; match the feature points of the ultraviolet preprocessed image and the visible light preprocessed image after preliminary positioning, and interpolate the visible light preprocessed image according to the matching result to process the visible light preprocessed image into an image corresponding to the position of the ultraviolet preprocessed image; fuse the ultraviolet preprocessed image and the visible light preprocessed image based on the nonlinear principal component analysis NSCT transform, and output the position information of the insulator defect; sharpen and enhance the fused image to obtain a fused image.

[0131] In some embodiments of the present application, in the above-mentioned device, the insulator defect detection deep learning model is trained according to the following method: constructing an insulator defect detection deep learning model, the insulator defect detection deep learning model includes: a pyramid network, the pyramid network includes multiple residual blocks and a pyramid pooling block, each of the residual blocks contains multiple convolutional layers and batch normalization layers, the pyramid pooling block includes multiple pooling layers of different sizes, and each of the pooling layers is connected to a convolutional layer; the top of the pyramid network is also linked to at least one fully connected layer; constructing a training sample set, the training sample set at least includes a training set, the training set includes multiple one-to-one corresponding ultraviolet images and visible light images, and the ultraviolet images and the visible light images are marked with insulator defect levels; inputting the training sample set into the insulator defect detection deep learning model, extracting image features of the training samples through the pyramid network and fusing them, and mapping the fused diversified features to a preset defect level through the fully connected layer to obtain the insulator defect level.

[0132] It can be understood that the above-mentioned joint detection device for insulator defects can implement the various steps of the joint detection method for insulator defects provided in the aforementioned embodiments, and the relevant explanations on the joint detection method for insulator defects are applicable to the joint detection device for insulator defects, which will not be repeated here.

[0133] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 6At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0134] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0135] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0136] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a joint detection device for insulator defects at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0137] Acquire an ultraviolet image of the insulator to be inspected, and preprocess the ultraviolet image to obtain an ultraviolet preprocessed image containing a discharge spot area;

[0138] Acquire a visible light image of the insulator to be inspected, perform preprocessing and feature value extraction on the visible light image, and obtain a visible light preprocessing image and visible light image feature values;

[0139] Based on the ultraviolet preprocessed image, constructing an ultraviolet discharge spectrum of the insulator, and determining a defect discharge level according to the ultraviolet discharge spectrum;

[0140] Based on the ultraviolet discharge spectrum and the visible light image characteristic value, the ultraviolet preprocessed image and the visible light preprocessed image are fused to obtain a fused image and location information of insulator defects;

[0141] A pre-trained insulator defect detection deep learning model is used to perform defect recognition on the fused image to obtain the insulator defect grade.

[0142] The above application Figure 5 The method performed by the joint detection device for insulator defects disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0143] The electronic device may also perform Figure 5 A method for implementing a combined detection device for insulator defects in a Figure 5 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0144] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 5 The method performed by the combined detection device for insulator defects in the illustrated embodiment is specifically used to perform:

[0145] Acquire an ultraviolet image of the insulator to be inspected, and preprocess the ultraviolet image to obtain an ultraviolet preprocessed image containing a discharge spot area;

[0146] Acquire a visible light image of the insulator to be inspected, perform preprocessing and feature value extraction on the visible light image, and obtain a visible light preprocessing image and visible light image feature values;

[0147] Based on the ultraviolet preprocessed image, constructing an ultraviolet discharge spectrum of the insulator, and determining a defect discharge level according to the ultraviolet discharge spectrum;

[0148] Based on the ultraviolet discharge spectrum and the visible light image characteristic value, the ultraviolet preprocessed image and the visible light preprocessed image are fused to obtain a fused image and location information of insulator defects;

[0149] A pre-trained insulator defect detection deep learning model is used to perform defect recognition on the fused image to obtain the insulator defect grade.

[0150] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0151] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0155] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0156] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0157] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0158] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0159] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A joint detection method for insulator defects, characterized in that: include: Acquire an ultraviolet image of the insulator to be inspected, and preprocess the ultraviolet image to obtain an ultraviolet preprocessed image containing a discharge spot area; Acquire a visible light image of the insulator to be inspected, perform preprocessing and feature value extraction on the visible light image, and obtain a visible light preprocessing image and visible light image feature values; Based on the ultraviolet preprocessed image, constructing an ultraviolet discharge spectrum of the insulator, and determining a defect discharge level according to the ultraviolet discharge spectrum; Based on the ultraviolet discharge spectrum and the visible light image characteristic value, the ultraviolet pre-processed image and the visible light pre-processed image are fused to obtain a fused image and location information of insulator defects; A pre-trained insulator defect detection deep learning model is used to perform defect recognition on the fused image to obtain the insulator defect grade.

2. The method according to claim 1, characterized in that Preprocessing the ultraviolet image includes: Performing grayscale processing on the ultraviolet image to obtain an ultraviolet grayscale image; Performing binarization conversion processing on the ultraviolet grayscale image to obtain an ultraviolet binary image; Performing background area cutting processing on the ultraviolet binary image to obtain an image containing the discharge spot area; The boundary of the discharge spot is determined by using a threshold segmentation or edge detection algorithm to obtain an ultraviolet preprocessed image.

3. The method according to claim 2, characterized in that The step of constructing an ultraviolet discharge spectrum of an insulator based on the ultraviolet preprocessed image comprises: Based on the ultraviolet preprocessed image, performing discharge analysis on the discharge spot area to extract discharge spectrum information related to the discharge activity; The spectrum features in the discharge spectrum information are analyzed, and representative characteristic values ​​of the discharge state of the insulator are selected to form the ultraviolet discharge spectrum of the insulator.

4. The method according to claim 1, characterized in that Preprocessing the visible light image includes: Before acquiring the visible light image, color temperature correction of the camera is performed using a white card, and exposure correction of the camera is performed using a gray card; After acquiring the visible light image, performing grayscale processing on the visible light image to obtain a visible light grayscale image; Performing enhancement and denoising processing on the visible light grayscale image; The seed region growing method is used to extract the insulator disk area and obtain a visible light preprocessed image containing the disk area.

5. The method according to claim 4, characterized in that The feature value extraction of the visible light image is performed according to the following method: Extract the mean value of the U component and the mean value of the V component of the disk area in the YUV color space in the visible light preprocessing image; Compare the absolute value of the difference between the mean values ​​of the U component of the color of the contaminated insulator disks of level 0 and level IV, and the absolute value of the difference between the mean values ​​of the V component of the color of the contaminated insulator disks of level 0 and level IV, and take the component corresponding to the larger absolute value as the visible light image feature value.

6. The method according to claim 1, characterized in that Determining the defect discharge level according to the ultraviolet discharge spectrum includes: Define fuzzy sets and membership functions to convert precise discharge time data into fuzzy values; A fuzzy logic inference algorithm is used to infer the defect discharge level based on the fuzzy value.

7. The method according to claim 1, characterized in that The step of fusing the ultraviolet preprocessed image and the visible light preprocessed image based on the ultraviolet discharge spectrum and the visible light image characteristic value to obtain a fused image and location information of insulator defects includes: Determining the position information of the insulator defect in the ultraviolet preprocessed image based on the characteristic value in the ultraviolet discharge spectrum, and preliminarily locating the insulator defect in the visible light preprocessed image according to the position information combined with the characteristic value of the visible light image; The gradient pyramid decomposition method is used to identify feature points, and the local feature matching method of grayscale statistics is used to remove feature points with the same feature vectors. Performing feature point matching on the UV pre-processed image and the visible light pre-processed image after preliminary positioning, and performing interpolation calculation on the visible light pre-processed image according to the matching result, so as to process the visible light pre-processed image into an image corresponding to the position of the UV pre-processed image; Based on nonlinear principal component analysis (NSCT) transformation, the ultraviolet preprocessed image and the visible light preprocessed image are fused, and the position information of the insulator defect is output; The fused image is sharpened and enhanced to obtain a fused image.

8. The method according to claim 1, characterized in that The insulator defect detection deep learning model is trained according to the following method: Constructing an insulator defect detection deep learning model, the insulator defect detection deep learning model comprising: a pyramid network, the pyramid network comprising a plurality of residual blocks and a pyramid pooling block, each of the residual blocks comprising a plurality of convolutional layers and a batch normalization layer, the pyramid pooling block comprising a plurality of pooling layers of different sizes, each of the pooling layers being followed by a convolutional layer; the top of the pyramid network is also connected to at least one fully connected layer; Constructing a training sample set, the training sample set at least comprising a training set, the training set comprising a plurality of pairs of one-to-one corresponding ultraviolet images and visible light images, the ultraviolet images and the visible light images being annotated with insulator defect levels; The training sample set is input into the insulator defect detection deep learning model, the image features of the training samples are extracted and fused through the pyramid network, and the fused diversified features are mapped to the preset defect level through the fully connected layer to obtain the insulator defect level.

9. A combined detection device for insulator defects, characterized in that: The device comprises: A first acquisition unit is used to acquire an ultraviolet image of the insulator to be inspected, and preprocess the ultraviolet image to obtain an ultraviolet preprocessed image containing a discharge spot area; A second acquisition unit is used to acquire a visible light image of the insulator to be inspected, preprocess the visible light image and extract feature values ​​to obtain a visible light preprocessed image and a visible light image feature value; A spectrum construction unit, used to construct an ultraviolet discharge spectrum of the insulator based on the ultraviolet preprocessed image, and determine the defect discharge level according to the ultraviolet discharge spectrum; A fusion unit, configured to fuse the ultraviolet preprocessed image and the visible light preprocessed image based on the ultraviolet discharge spectrum and the visible light image characteristic value, so as to obtain a fused image and location information of insulator defects; The prediction unit is used to use a pre-trained insulator defect detection deep learning model to perform defect recognition on the fused image to obtain the insulator defect level.

10. A computer-readable storage medium storing one or more programs, characterized in that: When the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device executes the joint detection method for insulator defects as claimed in any one of claims 1 to 8.

Images