Insulator fault detection method and device based on multispectral image

Abstract

The invention provides an insulator fault detection method and device based on a multispectral image. The method comprises the following steps: acquiring a visible light image, an infrared image and an ultraviolet image of an insulator through acquisition equipment according to a preset acquisition condition; respectively preprocessing the visible light image, the infrared image and the ultraviolet image to obtain a preprocessed visible light image, a preprocessed infrared image and a preprocessed ultraviolet image; performing spatial registration processing according to the preprocessed visible light image, the preprocessed infrared image and the preprocessed ultraviolet image to obtain a registered image; and performing fault detection on the insulator according to the preprocessed visible light image, the registration image and the trained insulator fault detection model to obtain a fault detection result. According to the invention, the accuracy of insulator fault detection can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of insulator fault detection, and also to an insulator fault detection method and device based on multispectral imaging. Background Art

[0002] Transmission line inspections are crucial for ensuring the safe and reliable operation of power grids, and insulator testing is a key component of this task. To ensure the safe and reliable operation of transmission lines, regular inspections of insulators along these lines are essential, along with the detection and timely repair of any faults. Therefore, insulator fault detection technology is a key aspect of power system operation and maintenance. To ensure the safe operation of the system, timely detection and accurate identification of insulator faults is crucial. Existing technologies have proposed the use of convolutional neural networks for insulator fault detection, but due to insufficient data volume and precision, insulator fault detection accuracy is poor. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an insulator fault detection method and device based on multispectral imaging to improve the accuracy of insulator fault detection.

[0004] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0005] A first aspect of the present invention provides an insulator fault detection method based on multispectral imaging, comprising:

[0006] According to the preset acquisition conditions, the visible light image, infrared image and ultraviolet image of the insulator are collected by the acquisition equipment;

[0007] Preprocessing the visible light image, the infrared image, and the ultraviolet image respectively to obtain a preprocessed visible light image, a preprocessed infrared image, and a preprocessed ultraviolet image;

[0008] performing spatial registration processing on the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image to obtain a registered image;

[0009] Fault detection is performed on the insulator based on the preprocessed visible light image, the registered image and the trained insulator fault detection model to obtain a fault detection result; the insulator fault detection model is obtained by training a preset network model based on the collected historical visible light images, historical infrared images and historical ultraviolet images of the insulator.

[0010] Optionally, preprocessing the visible light image, the infrared image, and the ultraviolet image respectively to obtain a preprocessed visible light image, a preprocessed infrared image, and a preprocessed ultraviolet image includes:

[0011] according to The visible light image is subjected to denoising to obtain a preprocessed visible light image; wherein G(x, y) is the preprocessed visible light image, σ is the standard deviation, is the normalization coefficient, (x, y) is the position of the Gaussian kernel on the visible light image plane, and the value range of x and y is [-3σ, 3σ];

[0012] According to I1(x,y)=G·[I raw (x, y)-O] performs correction processing on the infrared image to obtain a pre-processed infrared image; wherein I1(x, y) is the pre-processed infrared image, G is the gain, I raw (x, y) is the grayscale value of the pixel (x, y) in the original infrared image, and O is the offset;

[0013] According to I2(x,y)=I raw (x,y)-I dark (x, y) is corrected to obtain a pre-processed ultraviolet image; wherein, I2(x, y) is the pre-processed ultraviolet image, I raw (x,y) is the UV image, I dark (x,y) is the dark image.

[0014] Optionally, performing spatial registration processing on the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image to obtain a registered image includes:

[0015] performing feature extraction on the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image to obtain feature points;

[0016] Performing feature description on the feature points to obtain feature descriptors;

[0017] Determining a feature pair set according to the feature descriptor;

[0018] Image transformation is performed on the pre-processed infrared image and the pre-processed ultraviolet image according to the feature pair set to obtain a registered image.

[0019] Optionally, determining a feature pair set according to the feature descriptor includes:

[0020] According to the feature descriptor and Determine the distance value;

[0021] Determining a matching feature descriptor according to the feature descriptor and the distance value;

[0022] The matching feature descriptors are screened according to preset screening conditions to obtain a feature pair set.

[0023] Optionally, performing image transformation on the preprocessed infrared image and the preprocessed ultraviolet image according to the feature pair set to obtain a registered image includes:

[0024] Determining transformation parameters according to the feature pair set;

[0025] Determining a transformation matrix according to the transformation parameters and a preset matrix;

[0026] Image transformation is performed on the pre-processed infrared image and the pre-processed ultraviolet image according to the transformation matrix and the feature pair set to obtain a registered image.

[0027] Optionally, performing fault detection on the insulator according to the preprocessed visible light image, the registered image, and a trained insulator fault detection model to obtain a fault detection result includes:

[0028] Inputting the preprocessed visible light image and the registered image into a feature extraction layer of an insulator fault detection model for feature extraction to obtain feature data;

[0029] Inputting the feature data into a feature fusion layer of an insulator fault detection model for feature fusion to obtain fused feature data;

[0030] The fused feature data is input into a target detection layer of an insulator fault detection model to perform fault detection and obtain a fault detection result.

[0031] Optionally, the training process of the insulator fault detection model includes:

[0032] Collect historical visible light images, historical infrared images, and historical ultraviolet images of insulators;

[0033] Performing spatial registration on the historical visible light image, the historical infrared image, and the historical ultraviolet image to obtain a historical registered image;

[0034] Annotating the historical registration image to obtain a data sample;

[0035] The preset network model is trained according to the data samples to obtain an insulator fault detection model.

[0036] A second aspect of the present invention provides an insulator fault detection device based on multispectral imaging, comprising:

[0037] An acquisition module is used to acquire visible light images, infrared images, and ultraviolet images of the insulator through an acquisition device according to preset acquisition conditions;

[0038] A processing module is configured to preprocess the visible light image, the infrared image, and the ultraviolet image, respectively, to obtain a preprocessed visible light image, a preprocessed infrared image, and a preprocessed ultraviolet image; perform spatial registration processing on the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image to obtain a registered image; perform fault detection on the insulator based on the preprocessed visible light image, the registered image, and a trained insulator fault detection model to obtain a fault detection result; the insulator fault detection model is obtained by training a preset network model based on collected historical visible light images, historical infrared images, and historical ultraviolet images of the insulator.

[0039] According to a third aspect of the present invention, a computing device is provided, comprising: a processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to the first aspect is executed.

[0040] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions, which, when executed on a computer, causes the computer to execute the method described in the first aspect.

[0041] The above solution of the present invention includes at least the following beneficial effects:

[0042] The above-mentioned scheme of the present invention uses an acquisition device to acquire visible light images, infrared images and ultraviolet images of insulators according to preset acquisition conditions, and then preprocesses the visible light images, infrared images and ultraviolet images respectively to obtain preprocessed visible light images, preprocessed infrared images and preprocessed ultraviolet images, and then performs spatial registration processing on the preprocessed visible light images, preprocessed infrared images and preprocessed ultraviolet images to obtain registered images. Finally, fault detection is performed on the insulator based on the preprocessed visible light images, the registered images and a trained insulator fault detection model to obtain a fault detection result. The fault detection of the insulator is performed using multiple images, thereby improving the accuracy of insulator fault detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of an insulator fault detection method based on multispectral imaging in an embodiment of the present invention;

[0044] Figure 2 3 is a schematic structural diagram of an insulator fault detection device based on multispectral imaging in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0046] like Figure 1 As shown, an embodiment of the present invention proposes an insulator fault detection method based on multispectral imaging, comprising the following steps:

[0047] Step 101: collecting visible light images, infrared images, and ultraviolet images of the insulator using a collection device according to preset collection conditions;

[0048] Step 102, preprocessing the visible light image, the infrared image, and the ultraviolet image respectively to obtain a preprocessed visible light image, a preprocessed infrared image, and a preprocessed ultraviolet image;

[0049] Step 103, performing spatial registration processing on the pre-processed visible light image, the pre-processed infrared image, and the pre-processed ultraviolet image to obtain a registered image;

[0050] Step 104 , performing fault detection on the insulator based on the preprocessed visible light image, the registered image, and the trained insulator fault detection model to obtain a fault detection result; the insulator fault detection model is obtained by training a preset network model based on the collected historical visible light images, historical infrared images, and historical ultraviolet images of the insulator.

[0051] The insulator fault detection method based on multispectral images of an embodiment of the present invention uses an acquisition device to acquire visible light images, infrared images and ultraviolet images of the insulator according to preset acquisition conditions, and then preprocesses the visible light images, infrared images and ultraviolet images respectively to obtain preprocessed visible light images, preprocessed infrared images and preprocessed ultraviolet images. The preprocessed visible light images, preprocessed infrared images and preprocessed ultraviolet images are then spatially aligned to obtain aligned images. Finally, fault detection is performed on the insulator based on the preprocessed visible light images, the aligned images and a trained insulator fault detection model to obtain a fault detection result. The insulator fault detection is performed using multiple images, thereby improving the accuracy of insulator fault detection.

[0052] In an optional embodiment of the present invention, in step 101, a digital camera or a smart phone is used to capture a visible light image of the insulator under natural light or artificial lighting conditions, and the image is clear and the color is accurate; an infrared thermal imager is used to capture an infrared image of the insulator. The infrared thermal imager is not limited by lighting conditions, and the infrared image can display the temperature distribution of the insulator, which helps to detect potential overheating or fault areas; and an ultraviolet imager is used to capture an ultraviolet image of the insulator at night or in low light conditions. The ultraviolet imager can capture ultraviolet radiation emitted by objects, and the ultraviolet image can detect discharge phenomena in the insulator.

[0053] In an optional embodiment of the present invention, step 102 includes:

[0054] Step 1021, according to The visible light image is denoised to obtain a preprocessed visible light image; wherein G(x, y) is the preprocessed visible light image, and σ is the standard deviation, which is selected according to the noise level of the visible light image and the size of the target details. If the noise is Gaussian white noise, σ is 1 or 3 pixel units. is the normalization coefficient to ensure that the total energy of the Gaussian kernel is 1, (x, y) is the position of the Gaussian kernel on the visible light image plane, and the value range of x and y is [-3σ, 3σ] to ensure that the Gaussian kernel covers more than 99% of the energy;

[0055] Specifically, by performing denoising on the visible light image in the above manner, the surface structural features of the insulator in the visible light image can be enhanced and light / background interference can be suppressed.

[0056] Step 1022, according to I1(x,y)=G·[I raw (x, y)-O] performs correction processing on the infrared image to obtain a pre-processed infrared image; wherein I1(x, y) is the pre-processed infrared image, G is the gain, I raw (x, y) is the grayscale value of the pixel (x, y) in the original infrared image, and O is the offset;

[0057] Specifically, through correction processing, the stripe noise in the infrared image is eliminated, the contrast between the high-temperature area (such as the hot spot of the insulator) and the background is improved by more than 30%, and the temperature measurement error is reduced from ±5°C to within ±2°C.

[0058] Step 1023, according to I2(x,y)=I raw (x,y)-I dark (x, y) is used to correct the ultraviolet image to obtain a pre-processed ultraviolet image; wherein, I2(x, y) is the pre-processed ultraviolet image, I raw (x,y) is the UV image, I dark(x, y) is the dark image, that is, the UV image collected when the light source is turned off (or the detector is completely blocked).

[0059] Specifically, by preprocessing the visible light image, infrared image, and ultraviolet image respectively, the image quality and the accuracy of subsequent fault detection can be improved.

[0060] In an optional embodiment of the present invention, step 103 includes:

[0061] Step 1031 , performing feature extraction on the pre-processed visible light image, the pre-processed infrared image, and the pre-processed ultraviolet image to obtain feature points;

[0062] Specifically, significant feature points, such as edges, corners, and regions, are selected from the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image as feature points. These feature points serve as a basis for registration. In a specific embodiment, feature point extraction can also be performed in the following manner: a multi-scale Gaussian pyramid is constructed for each of the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image. Each layer of the image is blurred using a Gaussian kernel of a different scale to simulate the visual effect of the human eye observing an object from near to far. Gaussian blurred images of adjacent scales are subtracted to generate a Gaussian difference pyramid for detecting extreme points. In the Gaussian difference pyramid, each pixel is compared with 26 neighboring points at the same and adjacent scales. If it is an extreme point, it is retained. By fitting a three-dimensional quadratic function, the position error of the extreme point is eliminated, the positioning accuracy of the key point is improved, and noise interference is reduced. The gradient amplitude and direction are calculated within the neighborhood of the key point to generate a 36-dimensional directional histogram. The direction of the histogram peak is selected as the main direction. If multiple peaks exist, multiple key points, i.e., feature points, are generated.

[0063] Step 1032: performing feature description on the feature points to obtain feature descriptors;

[0064] Specifically, pixel pairs are randomly sampled within the neighborhood of the feature point of the preprocessed visible light image, and the grayscale values are compared to generate a binary string as the feature descriptor of the feature point of the preprocessed visible light image. Alternatively, the feature point of the preprocessed visible light image is divided into 4×4 sub-regions, and the 8-directional gradient histogram of each sub-region is counted to generate a 128-dimensional vector as the feature descriptor of the feature point of the preprocessed visible light image. By taking the feature point in the preprocessed infrared image as the center, the temperature gradient direction histogram (8 directions) within the 3×3 neighborhood is counted to generate a 24-dimensional vector as the feature descriptor of the feature point of the preprocessed infrared image. By counting the distribution histogram of photon counts (such as 0-10, 10-20, ..., 90-100 photons / pixel) within the neighborhood of the feature point in the preprocessed ultraviolet image, a 10-dimensional vector is generated as the feature descriptor of the feature point of the preprocessed ultraviolet image. Here, a feature descriptor is generated by describing each feature point, and the feature descriptor is used to measure the similarity between feature points.

[0065] Step 1033, determining a feature pair set according to the feature descriptor;

[0066] Specifically, the distance values between feature descriptors are first calculated, and then a feature pair set is obtained by screening according to the distance values to improve the registration accuracy.

[0067] Step 1034 : performing image transformation on the pre-processed infrared image and the pre-processed ultraviolet image according to the feature pair set to obtain a registered image.

[0068] Specifically, according to the deformation between the images, a suitable transformation model is selected, and the parameters of the transformation model are estimated using the matched feature point pairs (or matched feature descriptor pairs), and then the images are transformed to obtain the registered images.

[0069] In an optional embodiment of the present invention, step 1033 includes:

[0070] Step 10331, based on the feature descriptor and Determine the distance value;

[0071] Step 10332: Determine a matching feature descriptor based on the feature descriptor and the distance value;

[0072] Step 10333: Filter the matching feature descriptors according to preset filtering conditions to obtain a feature pair set.

[0073] Specifically, through Calculate the distance between each feature descriptor in the preprocessed visible light image, the feature descriptor in the preprocessed infrared image, and the feature descriptor in the preprocessed ultraviolet image. If the distance value is less than the preset distance value (such as the preset distance value between the preprocessed visible light image and the preprocessed infrared image is 0.7, the preset distance value between the preprocessed visible light image and the preprocessed ultraviolet image is 0.6, and the preset distance value between the preprocessed infrared image and the preprocessed ultraviolet image is 0.5), then the two feature descriptors are matched feature descriptors. Where d(u,v) is the distance value, u is the multidimensional attribute set vector of one feature descriptor, v is the multidimensional attribute set vector of another feature descriptor, and u i is the value of the eigenvector u in the i-th dimension, v i is the value of the feature vector v in the i-th dimension, and n is the number of dimensions. Here, the preset filtering condition is to delete matching feature descriptors with distance values less than a preset pixel value (e.g., 10 pixels), or to delete matching feature descriptors with distance values greater than a preset pixel value (e.g., 10 pixels). This results in a feature pair set. The appropriate preset filtering condition is selected based on actual needs. Feature pair sets include generating visible light-infrared and visible light-ultraviolet matching feature point descriptors.

[0074] In an optional embodiment of the present invention, step 1034 includes:

[0075] Step 10341, determining transformation parameters based on the feature pair set;

[0076] Specifically, through the feature pair set and x i ′ =ax i +by i +t x 、y i ′ =cx i +dy i +t y 、 Determine the transformation parameters a, b, c, d, t x , t y ,θ,s, here, the least squares method is used to directly solve the overdetermined equations (x i ′ =ax i +by i +t x 、y i ′ =cx i +dy i +t y). Here, the transformation parameters a, b, c, d must satisfy: a = s·cosθ, b = -s·sinθ, c = s·sinθ, d = s·cosθ. Among them, s is the proportional scaling factor, θ is the rotation angle, a, b, c, d are used to describe the rotation and scaling, t x , t y Used to describe translation.

[0077] Step 10342: Determine a transformation matrix based on the transformation parameters and a preset matrix;

[0078] Specifically, the preset matrix is Substituting the values of the transformation parameters into the preset matrix, the transformation matrix is obtained.

[0079] Step 10343: Perform image transformation on the pre-processed infrared image and the pre-processed ultraviolet image based on the feature pair set according to the transformation matrix to obtain a registered image.

[0080] Specifically, the pre-processed infrared image is directly transformed into the coordinate system of the visible light image through a forward transformation by applying the transformation matrix to obtain the registered visible light image. After obtaining the inverse of the transformation matrix, the pre-processed ultraviolet image is transformed into the coordinate system of the visible light image through an inverse transformation to obtain the registered ultraviolet image. Here, the registered images include the registered visible light image and the registered ultraviolet image.

[0081] In an optional embodiment of the present invention, in step 104, fault detection is performed on the insulator based on the preprocessed visible light image, the registered image, and the trained insulator fault detection model to obtain a fault detection result, including:

[0082] Step 10411: input the pre-processed visible light image and the registered image into a feature extraction layer of an insulator fault detection model to perform feature extraction to obtain feature data;

[0083] Specifically, the pre-processed visible light image provides intuitive features such as the spatial structure, texture, and color of the insulator (such as cracks on the surface of the porcelain bottle and the shape of the glass insulator shed); the infrared image in the registered image reflects the temperature distribution of the insulator and detects thermal anomalies caused by local discharge, contamination or aging (such as local high temperature areas); the ultraviolet image in the registered image captures the ultraviolet light signal generated by the corona discharge of the insulator and locates potential defects (such as discharge points and arc tracks). Here, the feature extraction layer can use the formula Feature extraction is performed on the pre-processed visible light image and the registered image to obtain feature data; where F out (i, j) is the value of the output feature map at position (i, j), and the feature data includes F out (i,j), F inis the input feature map (preprocessed visible light image or registered image), W(m,n) is the convolution kernel weight matrix, K is the convolution kernel size (such as 3×3 or 5×5), b is the bias,

[0084] Step 10412: Input the feature data into the feature fusion layer of the insulator fault detection model to perform feature fusion to obtain fused feature data;

[0085] Specifically, the feature fusion layer is implemented by w i =σ(MLP(AvgPool(F m ))+MaxPool(F m ))、F fused =w1·F1+w2·F2,+w3·F3 The feature data is fused to obtain fused feature data. i is the weight of the feature data, i=1,2,3; σ is the activation function, F m is the input feature data (feature map), MLP is a multi-layer perceptron, AvgPool is an average pooling operation, MaxPool is a maximum pooling operation, F fused It is the fusion feature data, F1, F2, and F3 are all feature data (for example, F1 is the feature data of the visible light image, F2 is the feature data of the infrared image, and F3 is the feature data of the ultraviolet image), w1 is the weight of F1, w2 is the weight of F2, and w3 is the weight of F3.

[0086] Step 10413: Input the fused feature data into the target detection layer of the insulator fault detection model to perform fault detection and obtain a fault detection result.

[0087] Specifically, a 3×3 convolution kernel is slid on the fused feature data (fused feature map) with a preset step size to generate multiple anchor frames (6 scale × 5 scale), and the Calculate the overlap between anchor frames; determine the target anchor frame based on the overlap between anchor frames and a preset screening threshold; and obtain the fault detection result based on the target anchor frame. If the overlap between multiple anchor frames is greater than the preset screening threshold and they all cover the same area, it is determined to be a true fault. In this case, the multiple anchor frames are target anchor frames. Based on the characteristics of the target anchor frame, an insulator fault can be determined. For example, if the target anchor frame has grayscale gradient > 50 within the anchor frame, aspect ratio > 3, and overlap with the insulator edge > 0.7, the corresponding insulator fault is a crack. If the target anchor frame has ultraviolet fluorescence intensity > 1000 counts and the intersection-and-union ratio of the infrared temperature anomaly area to the anchor frame > 0.5, the corresponding insulator fault is corona. If the target anchor frame has infrared temperature > insulator mean and the anchor frame area accounts for > 0.1 of the insulator body, the corresponding insulator fault is a thermal anomaly. Among them, Exp-IoU(A,B) is the overlap between anchor boxes, IoU(A,B) is the intersection-over-union ratio between anchor boxes, exp is the exponential function, and γ is an adjustable parameter.

[0088] In an optional embodiment of the present invention, the training process of the insulator fault detection model in step 104 includes:

[0089] Step 10421: collecting historical visible light images, historical infrared images, and historical ultraviolet images of the insulator;

[0090] Specifically, historical visible light images, historical infrared images, and historical ultraviolet images of the insulator within a preset time period can be acquired from the historical database.

[0091] Step 10422: spatially register the historical visible light image, the historical infrared image, and the historical ultraviolet image to obtain a historical registered image.

[0092] Specifically, the historical visible light image, historical infrared image, and historical ultraviolet image are first grayscale processed to convert the color image into a grayscale image, reducing the complexity of the data while retaining sufficient information for registration. The grayscale image is then denoised using methods such as filters to obtain a denoised image to reduce the impact of noise on the registration accuracy. If necessary, the denoised image can also be subjected to contrast enhancement, sharpening, and other processing to obtain a processed image to improve image quality. Significant feature points, such as edges, corners, and regions, are then selected from the processed image as feature points to serve as a basis for registration. Each feature point is then described to generate a feature descriptor, which is used to measure the similarity between feature points. By calculating the distance between the feature descriptors, the correspondence between the feature points in the visible light image, infrared image, and ultraviolet image is established to obtain matching feature points. Transformation model parameters are determined based on the matching feature points. Based on the transformation model parameters, the historical infrared image and historical ultraviolet image are transformed into the coordinate system of the historical visible light image to obtain a historically registered visible light image and a historically registered ultraviolet image.

[0093] Step 10423: annotate the historical registration image to obtain a data sample;

[0094] Specifically, the historical registration images are annotated, including the location (bounding box) and category (such as fault type) of the insulator fault. The annotated images are used as data samples for subsequent model training.

[0095] Step 10424: Train the preset network model based on the data sample to obtain an insulator fault detection model.

[0096] Specifically, the data samples are divided into a training set and a validation set in a ratio of 8:2. The training set is used to train the preset network model to obtain a training model. The validation set and preset indicators are used to evaluate the trained model. Based on the evaluation results, the trained model is adjusted and optimized to obtain an insulator fault detection model. The training method of this embodiment can improve the performance of the model and the accuracy of insulator fault detection.

[0097] A specific embodiment of the insulator fault detection method based on multispectral imaging according to an embodiment of the present invention includes:

[0098] Step 111, collecting visible light images, infrared images, and ultraviolet images of the insulator;

[0099] According to the acquisition equipment used, corresponding acquisition conditions are set, and visible light images, infrared images, and ultraviolet images of insulators are acquired under these conditions to improve image clarity and quality.

[0100] Step 112, image preprocessing;

[0101] The visible light image, infrared image and ultraviolet image are preprocessed by denoising and correction respectively to remove noise in the image, enhance the features of insulators in the image, and improve the accuracy of subsequent detection.

[0102] Step 113: image registration processing;

[0103] By extracting feature points of visible light images, infrared images, and ultraviolet images and pairing them, and then transforming different images into the same coordinate system, it is helpful to improve the processing efficiency of subsequent models.

[0104] Step 114: Insulator fault detection.

[0105] The trained insulator fault detection model is used to perform fault detection on insulators to obtain fault detection results, which improves the intelligence of the method and the effect of fault detection.

[0106] The insulator fault detection method based on multispectral images of an embodiment of the present invention fuses the different features of the insulator in the visible light image, infrared image and ultraviolet image, and then uses a trained insulator fault detection model to perform fault detection on the insulator. The fault detection results are relatively comprehensive and accurate.

[0107] like Figure 2 As shown, an embodiment of the present invention provides an insulator fault detection device 200 based on multispectral imaging, comprising:

[0108] The acquisition module 201 is used to acquire visible light images, infrared images, and ultraviolet images of the insulator through an acquisition device according to preset acquisition conditions;

[0109] The processing module 202 is used to preprocess the visible light image, the infrared image, and the ultraviolet image respectively to obtain a preprocessed visible light image, a preprocessed infrared image, and a preprocessed ultraviolet image; perform spatial registration processing on the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image to obtain a registered image; perform fault detection on the insulator based on the preprocessed visible light image, the registered image, and a trained insulator fault detection model to obtain a fault detection result; the insulator fault detection model is obtained by training a preset network model based on the collected historical visible light images, historical infrared images, and historical ultraviolet images of the insulator.

[0110] Optionally, preprocessing the visible light image, the infrared image, and the ultraviolet image respectively to obtain a preprocessed visible light image, a preprocessed infrared image, and a preprocessed ultraviolet image includes:

[0111] according to The visible light image is subjected to denoising to obtain a preprocessed visible light image; wherein G(x, y) is the preprocessed visible light image, σ is the standard deviation, is the normalization coefficient, (x, y) is the position of the Gaussian kernel on the visible light image plane, and the value range of x and y is [-3σ, 3σ];

[0112] According to I1(x,y)=G·[I raw (x, y)-O] is corrected to obtain a pre-processed infrared image; wherein I1(x, y) is the pre-processed infrared image, G is the gain, I raw (x, y) is the grayscale value of the pixel (x, y) in the original infrared image, and O is the offset;

[0113] According to I2(x,y)=I raw (x,y)-I dark (x, y) is corrected to obtain a pre-processed ultraviolet image; wherein, I2(x, y) is the pre-processed ultraviolet image, I raw (x,y) is the UV image, I dark (x,y) is the dark image.

[0114] Optionally, performing spatial registration processing on the ultraviolet image and the preprocessed visible light image, the preprocessed infrared image and the preprocessed ultraviolet image to obtain a registered image includes:

[0115] performing feature extraction on the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image to obtain feature points;

[0116] Performing feature description on the feature points to obtain feature descriptors;

[0117] Determining a feature pair set according to the feature descriptor;

[0118] Image transformation is performed on the pre-processed infrared image and the pre-processed ultraviolet image according to the feature pair set to obtain a registered image.

[0119] Optionally, determining a feature pair set according to the feature descriptor includes:

[0120] According to the feature descriptor and Determine the distance value;

[0121] Determining a matching feature descriptor according to the feature descriptor and the distance value;

[0122] The matching feature descriptors are screened according to preset screening conditions to obtain a feature pair set.

[0123] Optionally, performing image transformation on the preprocessed infrared image and the preprocessed ultraviolet image according to the feature pair set to obtain a registered image includes:

[0124] Determining transformation parameters according to the feature pair set;

[0125] Determining a transformation matrix according to the transformation parameters and a preset matrix;

[0126] Image transformation is performed on the pre-processed infrared image and the pre-processed ultraviolet image according to the transformation matrix and the feature pair set to obtain a registered image.

[0127] Optionally, performing fault detection on the insulator according to the preprocessed visible light image, the registered image, and a trained insulator fault detection model to obtain a fault detection result includes:

[0128] Inputting the preprocessed visible light image and the registered image into a feature extraction layer of an insulator fault detection model for feature extraction to obtain feature data;

[0129] Inputting the feature data into a feature fusion layer of an insulator fault detection model for feature fusion to obtain fused feature data;

[0130] The fused feature data is input into a target detection layer of an insulator fault detection model to perform fault detection and obtain a fault detection result.

[0131] Optionally, the training process of the insulator fault detection model includes:

[0132] Collect historical visible light images, historical infrared images, and historical ultraviolet images of insulators;

[0133] Performing spatial registration on the historical visible light image, the historical infrared image, and the historical ultraviolet image to obtain a historical registered image;

[0134] Annotating the historical registration image to obtain a data sample;

[0135] The preset network model is trained according to the data samples to obtain an insulator fault detection model.

[0136] The insulator fault detection device based on multispectral images of the embodiment of the present invention uses an acquisition device to acquire visible light images, infrared images and ultraviolet images of the insulator according to preset acquisition conditions, and then preprocesses the visible light images, infrared images and ultraviolet images respectively to obtain preprocessed visible light images, preprocessed infrared images and preprocessed ultraviolet images. The preprocessed visible light images, preprocessed infrared images and preprocessed ultraviolet images are then spatially aligned to obtain aligned images. Finally, fault detection is performed on the insulator based on the preprocessed visible light images, the aligned images and the trained insulator fault detection model to obtain a fault detection result. The insulator fault detection is performed using multiple images, thereby improving the accuracy of insulator fault detection.

[0137] It should be noted that the device is a device corresponding to the above method, and all implementations in the above method embodiment are applicable to the embodiment of the device and can achieve the same technical effects, which will not be described in detail in this embodiment.

[0138] An embodiment of the present invention further provides a computing device comprising: a processor and a memory storing a computer program. When the computer program is executed by the processor, the computer program performs the method described in any of the above embodiments. All implementations in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effects. These are not further described in this embodiment.

[0139] An embodiment of the present invention further provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described in any of the above embodiments. All implementations in the above method embodiments are applicable to the embodiments of the present invention and can achieve the same technical effects. These are not further described in this embodiment.

[0140] It should be noted that, in the apparatus and method of the present invention, it is apparent that each component or step can be decomposed and / or recombined. Such decomposition and / or recombination should be considered equivalent solutions of the present invention. Furthermore, the steps of performing the above series of processes can naturally be performed in chronological order according to the order described, but do not necessarily need to be performed in chronological order. Certain steps can be performed in parallel, interleaved, or independently of each other.

[0141] It should be noted that, in the above embodiments, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the implementation methods of the above embodiments is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0142] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for detecting insulator faults based on multispectral imaging, characterized in that: include: According to the preset acquisition conditions, the visible light image, infrared image and ultraviolet image of the insulator are collected by the acquisition equipment; Preprocessing the visible light image, the infrared image, and the ultraviolet image respectively to obtain a preprocessed visible light image, a preprocessed infrared image, and a preprocessed ultraviolet image; performing spatial registration processing on the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image to obtain a registered image; performing fault detection on the insulator according to the preprocessed visible light image, the registered image and the trained insulator fault detection model to obtain a fault detection result; The insulator fault detection model is obtained by training a preset network model based on collected historical visible light images, historical infrared images, and historical ultraviolet images of insulators.

2. The insulator fault detection method based on multispectral imaging according to claim 1, characterized in that: Preprocessing the visible light image, the infrared image, and the ultraviolet image respectively to obtain a preprocessed visible light image, a preprocessed infrared image, and a preprocessed ultraviolet image includes: according to The visible light image is subjected to denoising to obtain a preprocessed visible light image; wherein G(x, y) is the preprocessed visible light image, σ is the standard deviation, is the normalization coefficient, (x, y) is the position of the Gaussian kernel on the visible light image plane, and the value range of x and y is [-3σ, 3σ]; According to I1(x,y)=G·[I raw (x, y)-O] performs correction processing on the infrared image to obtain a pre-processed infrared image; wherein I1(x, y) is the pre-processed infrared image, G is the gain, I raw (x, y) is the grayscale value of the pixel (x, y) in the original infrared image, and O is the offset; According to I2(x,y)=I raw (x,y)-I dark (x, y) is used to correct the ultraviolet image to obtain a pre-processed ultraviolet image; wherein, I2(x, y) is the pre-processed ultraviolet image, I raw (x,y) is the UV image, I dark (x,y) is the dark image.

3. The insulator fault detection method based on multispectral imaging according to claim 1, characterized in that: Performing spatial registration processing on the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image to obtain a registered image includes: performing feature extraction on the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image to obtain feature points; Performing feature description on the feature points to obtain feature descriptors; Determining a feature pair set according to the feature descriptor; Image transformation is performed on the pre-processed infrared image and the pre-processed ultraviolet image according to the feature pair set to obtain a registered image.

4. The insulator fault detection method based on multispectral imaging according to claim 3, characterized in that: Determining a feature pair set according to the feature descriptor includes: According to the feature descriptor and Determine the distance value; Determining a matching feature descriptor according to the feature descriptor and the distance value; The matching feature descriptors are screened according to preset screening conditions to obtain a feature pair set.

5. The insulator fault detection method based on multispectral imaging according to claim 3, characterized in that: Performing image transformation on the preprocessed infrared image and the preprocessed ultraviolet image according to the feature pair set to obtain a registered image, comprising: Determining transformation parameters according to the feature pair set; Determining a transformation matrix according to the transformation parameters and a preset matrix; Image transformation is performed on the pre-processed infrared image and the pre-processed ultraviolet image according to the transformation matrix and the feature pair set to obtain a registered image.

6. The insulator fault detection method based on multispectral imaging according to claim 1, characterized in that: Performing fault detection on the insulator according to the preprocessed visible light image, the registered image, and the trained insulator fault detection model to obtain a fault detection result, including: Inputting the preprocessed visible light image and the registered image into a feature extraction layer of an insulator fault detection model for feature extraction to obtain feature data; Inputting the feature data into a feature fusion layer of an insulator fault detection model for feature fusion to obtain fused feature data; The fused feature data is input into a target detection layer of an insulator fault detection model to perform fault detection and obtain a fault detection result.

7. The insulator fault detection method based on multispectral imaging according to claim 1, characterized in that: The training process of the insulator fault detection model includes: Collect historical visible light images, historical infrared images, and historical ultraviolet images of insulators; Performing spatial registration on the historical visible light image, the historical infrared image, and the historical ultraviolet image to obtain a historical registered image; Annotating the historical registration image to obtain a data sample; The preset network model is trained according to the data samples to obtain an insulator fault detection model.

8. An insulator fault detection device based on multispectral imaging, characterized in that: include: An acquisition module is used to acquire visible light images, infrared images, and ultraviolet images of the insulator through an acquisition device according to preset acquisition conditions; a processing module, configured to preprocess the visible light image, the infrared image, and the ultraviolet image, respectively, to obtain a preprocessed visible light image, a preprocessed infrared image, and a preprocessed ultraviolet image; performing spatial registration processing on the preprocessed visible light image, the preprocessed infrared image, and the preprocessed ultraviolet image to obtain a registered image; performing fault detection on the insulator according to the preprocessed visible light image, the registered image and the trained insulator fault detection model to obtain a fault detection result; The insulator fault detection model is obtained by training a preset network model based on collected historical visible light images, historical infrared images, and historical ultraviolet images of insulators.

9. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is performed.

10. A computer-readable storage medium, characterized in that The device stores instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 7.

Images