Computer vision-based rapid detection method, device and system for electric cabinet faults

Abstract

The present invention discloses a method, device and system for rapid detection of electric cabinet faults based on computer vision. The device includes an image acquisition module, a noise separation module, a fault area marking module, an optimal threshold determination module, a fault area determination module, a connected area segmentation module, a real fault area determination module, a model training module and a fault detection module. The present invention belongs to the technical field of electric cabinet fault detection, and specifically refers to a method, device and system for rapid detection of electric cabinet faults based on computer vision. The present solution can quickly and effectively separate high-frequency stripe noise and low-frequency thermal radiation noise from infrared images, effectively improving the quality of infrared images, and thus being suitable for occasions with strict requirements on image quality and processing speed. At the same time, the present solution can accurately locate the fault area, and has extremely high versatility, thereby greatly improving the accuracy and efficiency of fault detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of electric cabinet fault detection, and specifically refers to a method, device and system for rapid detection of electric cabinet faults based on computer vision. Background Art

[0002] The computer vision-based rapid detection method for electrical cabinet faults is a detection technology that combines computer vision technology with deep learning technology and uses infrared image acquisition to quickly detect internal faults in electrical cabinets, which helps to improve the reliability and safety of equipment operation.

[0003] However, the existing computer vision-based rapid detection methods for electrical cabinet faults still have many defects:

[0004] 1. When separating noise, the existing noise separation methods usually only focus on high-frequency stripe noise, while ignoring the impact of low-frequency thermal radiation noise on image quality. At the same time, the existing noise separation methods lack reasonable prior information to guide the noise structure, and usually directly solve the optimal estimate by constructing a loss function. This method requires repeated adjustment of parameters, which will not only greatly increase the running time of the algorithm, but also may cause the parameters to fall into local optimality or parameter divergence. Therefore, this method is not suitable for occasions with strict requirements on image quality and processing speed.

[0005] 2. Due to the wide variety of components, parts and equipment inside the distribution cabinet, the existing fault detection methods generally have technical problems such as difficulty in accurately locating the fault area and low versatility, resulting in low accuracy and efficiency of fault detection.

[0006] 3. The existing fault detection method combined with deep learning technology is prone to technical problems of gradient vanishing and gradient exploding, which affects the accuracy and efficiency of fault detection. Summary of the invention

[0007] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a method, device and system for rapid detection of electric cabinet faults based on computer vision. In view of the technical problems that the existing noise separation methods usually only focus on high-frequency stripe noise, but ignore the impact of low-frequency thermal radiation noise on image quality, and lack reasonable prior information to guide the noise structure, and usually directly solve the optimal estimate by constructing a loss function, resulting in a long running time of the algorithm and easy to cause parameters to fall into local optimal or parameter divergence, this scheme realizes two-dimensional discrete wavelet transform through digital filters and downsamplers, and obtains an approximate low-frequency component and high-frequency component through multiple iterations. image and detailed image, calculate the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform according to the approximate image and the detailed image containing the low-frequency component and the high-frequency component, use a cluster analysis algorithm to separate the high-frequency stripe noise according to the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform, and obtain an infrared image from which the high-frequency stripe noise is removed, and obtain a low-frequency component by performing a wavelet domain transform on the infrared image from which the high-frequency stripe noise is removed, and approximate the low-frequency component by using a Bezier surface fitting algorithm to obtain a Bezier surface Introduce the noise intensity coefficient λ and according to the Bezier surface A correction model is constructed, and the noise intensity coefficient λ is determined by minimizing the energy function. In the range of [0,1], a direct search method with a step size of 0.1 is used to obtain the optimal noise intensity coefficient λ. The optimal noise intensity coefficient λ is substituted into the correction model to remove low-frequency thermal radiation noise, and an infrared image with high-frequency stripe noise and low-frequency thermal radiation noise removed is obtained. Through the above operation, high-frequency stripe noise and low-frequency thermal radiation noise can be effectively separated from the infrared image, and the quality of the infrared image is effectively improved, thereby solving the technical problem that the existing noise separation method usually only focuses on high-frequency stripe noise and ignores the influence of low-frequency thermal radiation noise on image quality; at the same time, the above operation can simply and efficiently eliminate high-frequency stripe noise and low-frequency thermal radiation noise without the need for reasonable prior information to guide the noise structure, and there is no need to directly solve the optimal estimate by constructing a loss function, thereby The technical problems that the algorithm has a long running time and is prone to parameters falling into local optimum or parameter divergence due to the lack of reasonable prior information to guide the noise structure and the direct solution of the optimal estimate by constructing a loss function are solved. Therefore, this scheme is suitable for occasions with strict requirements on image quality and processing speed. In view of the technical problems that the existing fault detection methods generally have difficulty in accurately locating the fault area and the technical problems of low versatility due to the wide variety of components, parts and equipment inside the distribution cabinet, this scheme converts the infrared image with high-frequency stripe noise and low-frequency thermal radiation noise removed into an HSV color space image, performs clustering and normalization operations on the HSV color space image, and marks the pixels that meet the constraints as fault area pixels to obtain the HSV color space fault image, and determines the optimal threshold T of the V channel of the HSV color space fault image through the OTSU algorithm. M , and according to the optimal threshold T M Perform binarization to obtain a binary V channel image, perform morphological closing operation on the binary V channel image, and obtain the fault area R S , the binary V channel image is divided into multiple connected regions R0 by the connected component analysis algorithm, and each connected region R0 is traversed. If the connected region R0 is consistent with the fault region R S The intersection area is greater than 0.8 times the fault area R Sarea, the connected area is set as the real fault area to obtain the real fault area image, and a real fault area image dataset is constructed according to the real fault area image and the category of the real fault area image. Through the above operation, the fault area can be accurately located, and the versatility is extremely high, thereby greatly improving the accuracy and efficiency of fault detection; in view of the technical problem that the existing fault detection method combined with deep learning technology is prone to gradient vanishing and gradient explosion, this scheme inputs the real fault area image dataset into the R-FCN network model that introduces the residual network structure, and trains the model through the online difficult example mining method to obtain a fault detection model, thereby solving the technical problem that the existing fault detection method combined with deep learning technology is prone to gradient vanishing and gradient explosion, and improving the accuracy and efficiency of fault detection.

[0008] The technical solution adopted by the present invention is as follows: The method for rapid detection of electric cabinet faults based on computer vision provided by the present invention comprises:

[0009] Step S1: image acquisition and construction of an image data set, specifically, acquisition of various types of infrared images of the interior of a distribution cabinet, and construction of an infrared image data set of the interior of a distribution cabinet using the infrared images of the interior of a distribution cabinet and the categories of the infrared images of the interior of a distribution cabinet;

[0010] Step S2: noise separation, specifically, separating high-frequency stripe noise and low-frequency thermal radiation noise from the infrared image inside the power distribution cabinet in the infrared image data set inside the power distribution cabinet, to obtain an infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed;

[0011] Step S3: marking the fault area, specifically, converting the infrared image from which high-frequency stripe noise and low-frequency thermal radiation noise are removed into an HSV color space image, performing clustering and normalization operations on the HSV color space image, and marking the pixels that meet the constraint conditions as the pixels of the fault area, thereby obtaining an HSV color space fault image;

[0012] Step S4: Determine the optimal threshold, specifically, determine the optimal threshold T of the V channel of the fault image in the HSV color space by using the OTSU algorithm M ;

[0013] Step S5: Determine the fault area, specifically, according to the optimal threshold T M Perform binarization to obtain a binary V channel image, perform morphological closing operation on the binary V channel image, and obtain the fault area R S ;

[0014] Step S6: connected region segmentation, specifically, segmenting the binary V channel image into multiple connected regions R0 by a connected component analysis algorithm;

[0015] Step S7: Determine the real fault area, specifically, traverse each connected area R0, if the connected area R0 is S The intersection area is greater than 0.8 times the fault area R S , then the connected area is set as the real fault area to obtain the real fault area image, and a real fault area image dataset is constructed according to the real fault area image and the category of the real fault area image;

[0016] Step S8: Model training, specifically, inputting the real fault area image dataset into the R-FCN network model with the residual network structure for training to obtain a fault detection model;

[0017] The R-FCN network model introducing the residual network structure includes a pre-trained convolutional neural network, a full convolutional network, a residual network structure, a ROI pooling layer, a classification layer and a regression layer;

[0018] The pre-trained convolutional neural network extracts a feature map of an input real fault area image dataset and sends the feature map to a full convolutional network;

[0019] The fully convolutional network converts the feature map into a dense category score map and a position regression map by introducing a residual network structure, sends the dense category score map to the classification layer, and sends the position regression map to the regression layer;

[0020] The fully convolutional network generates several ROIs by sliding a fixed-size window on the feature map, where the ROI is a region of interest;

[0021] The ROI pooling layer maps the ROI to obtain a pooled ROI feature map, and sends the pooled ROI feature map to the parallel classification layer and regression layer;

[0022] The classification layer and the regression layer are both fully connected layers, and the classification layer outputs the probability distribution and true label of each ROI belonging to different categories;

[0023] The regression layer outputs the bounding box coordinate adjustment and true label of each ROI;

[0024] Step S9: Fault detection, specifically, performing fault detection through a fault detection model to obtain the fault type, fault level and fault location.

[0025] As a further improvement of this solution, in step S2, the noise separation step includes:

[0026] Step S21: Separating high-frequency stripe noise;

[0027] Step S22: Separating low-frequency thermal radiation noise;

[0028] In step S21, the step of separating high-frequency stripe noise includes:

[0029] Step S211: performing a two-dimensional discrete wavelet transform on the infrared image inside the power distribution cabinet, specifically, implementing a two-dimensional discrete wavelet transform through a digital filter and a downsampler, and obtaining an approximate image and a detailed image containing low-frequency components and high-frequency components through multiple iterations;

[0030] Step S212: Calculate the wavelet coefficients, specifically, calculate the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component, and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform according to the approximate image and the detailed image containing the low-frequency component and the high-frequency component, and the calculation formula of the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component, and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform is:

[0031]

[0032]

[0033]

[0034] In the formula, and They represent the wavelet coefficients of the vertical component, the horizontal component, and the diagonal component in the j-th scale space of the wavelet transform, respectively. M represents the number of rows of the image, N represents the number of columns of the image, m represents the row index of the image, and n represents the column index of the image. Indicates the summation of all pixels in the image. Represents the approximate image of the previous scale space of the j-th scale space, and They represent the wavelet function of the vertical component, the wavelet function of the horizontal component and the wavelet function of the diagonal component in the j-th scale space of the wavelet transform respectively;

[0035] Step S213: separating the high-frequency stripe noise according to the wavelet coefficients, specifically, using a cluster analysis algorithm to separate the high-frequency stripe noise according to the wavelet coefficients of the vertical component, the horizontal component and the diagonal component in each scale space of the wavelet transform, to obtain an infrared image with the high-frequency stripe noise removed;

[0036] In step S213, the step of separating high-frequency stripe noise according to wavelet coefficients includes:

[0037] Step S2131: clustering the wavelet coefficients of the vertical component, the horizontal component and the diagonal component in each scale space of the wavelet transform, classifying similar wavelet coefficients into the same category, and obtaining a clustering result;

[0038] Step S2132: extracting the wavelet coefficients belonging to the high-frequency stripe noise according to the clustering result, and obtaining a high-frequency stripe noise image;

[0039] Step S2133: performing a subtraction operation on the infrared image inside the power distribution cabinet and the high-frequency stripe noise image to obtain an infrared image with the high-frequency stripe noise removed;

[0040] In step S22, the step of separating low-frequency thermal radiation noise includes:

[0041] Step S221: performing wavelet domain transformation on the infrared image after removing high-frequency stripe noise to obtain a low-frequency component;

[0042] Step S222: Approximately estimate the low-frequency component using a Bezier surface fitting algorithm to obtain a Bezier surface

[0043] Step S223: Introduce the noise intensity coefficient λ and calculate the noise intensity coefficient according to the Bezier surface. A correction model is constructed, and the formula of the correction model is:

[0044]

[0045] In the formula, b represents the corrected image, λ represents the noise intensity coefficient, and × represents the multiplication operation. represents a Bezier surface;

[0046] Step S224: Calculate the noise intensity coefficient λ. Specifically, determine the noise intensity coefficient λ by minimizing the energy function. The calculation formula of the noise intensity coefficient λ is:

[0047] λ = argmin[J(λ)];

[0048] J(λ)={std[mean(I-λb)]-grad(I-λb)};

[0049] Where, λ represents the noise intensity coefficient, J(λ) represents the energy function of λ, argmin[J(λ)] represents the energy function that minimizes λ, b represents the corrected image, mean represents the calculation of the mean value of each column of the image to obtain a one-dimensional array, std represents the calculation of the standard deviation of the one-dimensional array, grad represents the calculation of the gradient of the image, and I represents the infrared image with high-frequency stripe noise removed;

[0050] Step S225: in the range of [0,1], a direct search method with a step size of 0.1 is used to obtain the optimal noise intensity coefficient λ;

[0051] Step S226: Substituting the optimal noise intensity coefficient λ into the correction model to remove the low-frequency thermal radiation noise, thereby obtaining an infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed;

[0052] In step S222, the step of approximating the low-frequency component by using the Bezier surface fitting algorithm includes:

[0053] Step S2221: define the value of the Bezier surface, the formula for defining the value of the Bezier surface is:

[0054]

[0055]

[0056] Where, the parameters u and v represent the position coordinates of the Bezier surface in the u direction and in the v direction, respectively, and the value range of u and v is [0,1]. b(u,v) represents the value of the point with the position coordinates (u,v) on the Bezier surface. i,j represents the coordinates of the control points of the Bezier surface, where i and j represent the index of the control point of the Bezier surface in the u direction and the index in the v direction, respectively, m and n represent the degree of the Bezier surface in the u direction and the degree of the Bezier surface in the v direction, respectively, t represents the interpolation position of the Bernstein polynomial, B i,m (u) represents the Bernstein polynomial of degree m on the Bezier surface in the u direction, B j,n (v) represents the Bernstein polynomial of degree n on the Bezier surface in the v direction, represents the number of combinations of selecting i elements from n elements, B i,n (t) represents the Bernstein polynomial of degree n at the interpolation position t;

[0057] Step S2222: constructing a matrix equation according to a formula defining the value of the Bezier surface;

[0058] Step S2223: Solve the matrix equation by the least square method to determine the control point coordinates P of the Bezier surface and obtain the Bezier surface

[0059] As a further improvement of this solution, in step S3, the step of marking the fault area includes:

[0060] Step S31: converting the infrared image from which high-frequency stripe noise and low-frequency thermal radiation noise are removed into an HSV color space image;

[0061] Step S32: Perform a clustering operation on the HSV color space image to obtain the clustered HSV color space image;

[0062] Step S33: Normalize the clustered HSV color space image to ensure that the value of the H channel of the clustered HSV color space image remains within the range of [0, 360], and update the value of the H channel of the clustered HSV color space image to obtain the normalized HSV color space image;

[0063] Step S34: Set the constraint conditions: 10 < H < 150, 10 < S < 100, and 200 < V < 255, where H represents the value of the H channel of the HSV color space image, S represents the value of the S channel of the HSV color space image, and V represents the value of the V channel of the HSV color space image. Traverse all pixel points of the normalized HSV color space image. If the values of the H channel, S channel, and V channel of the pixel point all satisfy the constraint conditions, then set the values of the H channel, S channel, and V channel of the pixel point that satisfies the constraint conditions to 0, and mark the pixel point that satisfies the constraint conditions as a fault area pixel point to obtain the HSV color space fault image;

[0064] In step S32, the step of performing a clustering operation on the HSV color space image includes:

[0065] Step S321: Initialize the clustering center points of the HSV color space image so that each clustering center point is evenly distributed in the HSV color space image according to the distance S;

[0066] Step S322: Calculate the gradient of the 3×3 neighborhood around each clustering center point, and select the pixel point with the minimum gradient as the new clustering center point within the 3×3 neighborhood;

[0067] Step S323: Calculate the distance metric D from each pixel point to each clustering center point, and assign the pixel point to the clustering center point with the minimum distance metric D.

[0068] As a further improvement of this solution, in step S4, the step of determining the optimal threshold includes:

[0069] Step S41: Count the number of pixel points and calculate the probability distribution. Specifically, count the number of pixel points of each gray level in the V channel of the HSV color space fault image, and calculate the probability distribution of each gray level in the V channel of the HSV color space fault image. The calculation formula for the probability distribution of each gray level in the V channel of the HSV color space fault image is:

[0070]

[0071] In the formula, p i represents the probability distribution of the i-th gray level in the V channel of the fault image in the HSV color space, N i represents the number of pixels of the i-th gray level, and n represents the number of pixels in the V channel of the fault image in the HSV color space;

[0072] Step S42: Calculate the optimal threshold T M ;

[0073] In step S42, the optimal threshold T is calculated. M The steps include:

[0074] Step S421: presetting the maximum value of the inter-class variance to 0, and setting the inter-class variance threshold T corresponding to the maximum value of the inter-class variance;

[0075] Step S422: Calculate the average gray value of the V channel of the fault image in the HSV color space;

[0076] Step S423: Calculate the probability distribution of the foreground and the background at each gray level. The calculation formula for the probability distribution of the foreground and the background at each gray level is:

[0077]

[0078] p1(T)=1-p0(T);

[0079] In the formula, p0(T) represents the probability distribution of the foreground at each gray level, p1(T) represents the probability distribution of the background at each gray level, T represents the inter-class variance threshold corresponding to the maximum inter-class variance, and N i represents the number of pixels of the i-th gray level, N represents the number of pixels in the V channel of the fault image in the HSV color space, and i represents the index of the gray level;

[0080] Step S424: Calculate the average grayscale value of the foreground and the average grayscale value of the background at each grayscale level. The calculation formula for the average grayscale value of the foreground and the average grayscale value of the background at each grayscale level is:

[0081]

[0082]

[0083] Where μ0(T) represents the average gray value of the foreground at each gray level, μ1(T) represents the average gray value of the background at each gray level, m represents the average gray value of the V channel of the fault image in the HSV color space, p0(T) represents the probability distribution of the foreground at each gray level, p1(T) represents the probability distribution of the background at each gray level, T represents the inter-class variance threshold, and Ni represents the number of pixels of the i-th gray level, N represents the number of pixels in the fault image in the HSV color space, and i represents the index of the gray level;

[0084] Step S425: Calculate the inter-class variance. If the inter-class variance is greater than the maximum inter-class variance, update the maximum inter-class variance and the inter-class variance threshold T. The calculation formula of the inter-class variance is:

[0085]

[0086] In the formula, represents the inter-class variance, m represents the average gray value of the fault image in the HSV color space, p0(T) represents the probability distribution of the foreground at each gray level, p1(T) represents the probability distribution of the background at each gray level, T represents the inter-class variance threshold, μ0(T) represents the average gray value of the foreground at each gray level, and μ1(T) represents the average gray value of the background at each gray level;

[0087] Step S426: Set the inter-class variance threshold T corresponding to the maximum inter-class variance value as the optimal threshold T M .

[0088] As a further improvement of this solution, in step S5, the step of determining the fault area includes:

[0089] Step S51: According to the optimal threshold T M Binarizing the V channel of the HSV color space fault image to obtain a binary V channel image, wherein the binary V channel image contains pixel points in the fault area;

[0090] Step S52: Connect the pixels of the fault region of the binary V channel image by morphological closing operation to obtain the fault region R S .

[0091] As a further improvement of this solution, in step S8, the model training step includes:

[0092] Step S81: extracting a feature map of an input real fault area image dataset through a pre-trained convolutional neural network;

[0093] Step S82: introducing a residual network structure into the full convolutional network, and converting the feature map into a dense category score map and a position regression map through the full convolutional network with the residual network structure;

[0094] Step S83: Generate several ROIs by sliding a fixed-size window on the feature map through the full convolutional network;

[0095] Step S84: Mapping the ROI through the ROI pooling layer to obtain a pooled ROI feature map;

[0096] Step S85: Output the probability distribution and true label of each ROI belonging to different categories through the classification layer, which is used to determine whether the ROI contains the fault area, and output the bounding box coordinate adjustment and true label of each ROI through the regression layer, which is used to accurately locate the position of the fault area;

[0097] Step S86: Perform model training by an online hard example mining method according to the probability distribution and true label of each ROI belonging to different categories output by the classification layer, and the bounding box coordinate adjustment and true label of each ROI output by the regression layer;

[0098] Step S87: Repeat steps S81 to S87 until a convergence condition is reached to obtain a fault detection model;

[0099] In step S87, the step of performing model training by using the online hard example mining method includes:

[0100] Step S871: During each iteration of training, the classification loss of each ROI is calculated according to the probability distribution of each ROI belonging to different categories and the true label output by the classification layer;

[0101] Step S872: pre-set a classification threshold, and set the ROI with a classification loss greater than the classification threshold as a ROI hard case;

[0102] Step S873: input the ROI difficult examples and the true labels corresponding to the ROI difficult examples into the regression layer, and calculate the regression loss;

[0103] Step S874: Add the classification loss of the ROI difficult example to the classification loss regression loss of the ROI difficult example to obtain the total loss of the ROI difficult example, and update the parameters of the model through the optimization algorithm to minimize the total loss of the ROI difficult example.

[0104] The computer vision-based electric cabinet fault rapid detection device provided by the present invention comprises an image acquisition module, a noise separation module, a fault area marking module, an optimal threshold determination module, a fault area determination module, a connected area segmentation module, a real fault area determination module, a model training module and a fault detection module;

[0105] The image acquisition module is used for image acquisition and constructing an image data set, specifically, acquiring various types of infrared images of the inside of the distribution cabinet, and constructing an infrared image data set of the inside of the distribution cabinet using the infrared images of the inside of the distribution cabinet and the categories of the infrared images of the inside of the distribution cabinet, and sending the infrared image data set of the inside of the distribution cabinet to the noise separation module;

[0106] The noise separation module is used for noise separation, specifically, separating high-frequency stripe noise and low-frequency thermal radiation noise from the infrared image inside the power distribution cabinet in the infrared image data set inside the power distribution cabinet, obtaining an infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed, and sending the infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed to the fault area marking module;

[0107] The fault area marking module is used for fault area marking, specifically, converting the infrared image from which high-frequency stripe noise and low-frequency thermal radiation noise are removed into an HSV color space image, performing clustering and normalization operations on the HSV color space image, marking pixels that meet the constraint conditions as pixels of the fault area, obtaining an HSV color space fault image, and sending the HSV color space fault image to the optimal threshold determination module;

[0108] The optimal threshold determination module is used to determine the optimal threshold, specifically, to determine the optimal threshold T of the V channel of the fault image in the HSV color space by using the OTSU algorithm. M , and the optimal threshold T M Send to the fault area determination module;

[0109] The fault area determination module is used to determine the fault area, specifically, according to the optimal threshold T M Perform binarization to obtain a binary V channel image, perform morphological closing operation on the binary V channel image, and obtain the fault area R S , and send the binary V channel image to the connected region segmentation module, and segment the fault region R S Send to the real fault area determination module;

[0110] The connected region segmentation module is used for connected region segmentation, specifically, segmenting the binary V channel image into multiple connected regions R0 through a connected component analysis algorithm, and sending the multiple connected regions R0 to the real fault region determination module;

[0111] The real fault area determination module is used to determine the real fault area. Specifically, each connected area R0 is traversed. If the connected area R0 is consistent with the fault area R S The intersection area is greater than 0.8 times the fault area R S , the connected area is set as the real fault area to obtain the real fault area image, a real fault area image dataset is constructed according to the real fault area image and the category of the real fault area image, and the real fault area image dataset is sent to the model training module;

[0112] The model training module is used for model training, specifically, inputting a real fault area image dataset into an R-FCN network model that introduces a residual network structure for training to obtain a fault detection model, and sending the fault detection model to the fault detection module;

[0113] The fault detection module is used for fault detection, specifically, performing fault detection through a fault detection model to obtain the fault type, fault level and fault location.

[0114] The computer vision-based electric cabinet fault rapid detection system provided by the present invention comprises a memory and a processor, and the processor executes a computer program stored in the memory.

[0115] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0116] (1) In view of the technical problem that the existing noise separation methods usually only focus on high-frequency stripe noise and ignore the impact of low-frequency thermal radiation noise on image quality, and the lack of reasonable prior information to guide the noise structure, and the technical problem that the algorithm usually runs for a long time and easily causes parameters to fall into local optimality or parameter divergence by directly solving the optimal estimate by constructing a loss function, this scheme implements two-dimensional discrete wavelet transform through digital filters and downsamplers, and obtains approximate images and detailed images containing low-frequency components and high-frequency components through multiple iterations. According to the approximate images and detailed images containing low-frequency components and high-frequency components, the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component, and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform are calculated. According to the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component, and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform, a cluster analysis algorithm is used to separate the high-frequency stripe noise to obtain an infrared image with the high-frequency stripe noise removed, and a low-frequency component is obtained by performing a wavelet domain transform on the infrared image with the high-frequency stripe noise removed. The low-frequency component is approximately estimated by a Bezier surface fitting algorithm to obtain a Bezier surface Introduce the noise intensity coefficient λ and according to the Bezier surface A correction model is constructed, and the noise intensity coefficient λ is determined by minimizing the energy function. In the range of [0,1], a direct search method with a step size of 0.1 is used to obtain the optimal noise intensity coefficient λ. The optimal noise intensity coefficient λ is substituted into the correction model to remove the low-frequency thermal radiation noise, and an infrared image with high-frequency stripe noise and low-frequency thermal radiation noise removed is obtained. Through the above operation, the high-frequency stripe noise and the low-frequency thermal radiation noise can be effectively separated from the infrared image, and the quality of the infrared image is effectively improved, thereby solving the technical problem that the existing noise separation method usually only focuses on the high-frequency stripe noise and ignores the influence of the low-frequency thermal radiation noise on the image quality; at the same time, the above operation can simply and efficiently eliminate the high-frequency stripe noise and the low-frequency thermal radiation noise without the need for reasonable prior information to guide the noise structure, and does not need to directly solve the optimal estimate by constructing a loss function, thereby solving the technical problem that the algorithm has a long running time and is prone to parameters falling into local optimality or parameter divergence due to the lack of reasonable prior information to guide the noise structure and the direct solution of the optimal estimate by constructing a loss function. Therefore, this scheme is suitable for occasions with strict requirements on image quality and processing speed.

[0117] (2) Due to the wide variety of components, parts and equipment inside the distribution cabinet, the existing fault detection methods generally have the technical problem of difficulty in accurately locating the fault area and the technical problem of low versatility. This scheme converts the infrared image that removes high-frequency stripe noise and low-frequency thermal radiation noise into an HSV color space image, performs clustering and normalization operations on the HSV color space image, and marks the pixels that meet the constraints as the pixels of the fault area to obtain the HSV color space fault image. The OTSU algorithm is used to determine the optimal threshold T of the V channel of the HSV color space fault image. M , and according to the optimal threshold T M Perform binarization to obtain a binary V channel image, perform morphological closing operation on the binary V channel image, and obtain the fault area R S , the binary V channel image is divided into multiple connected regions R0 by the connected component analysis algorithm, and each connected region R0 is traversed. If the connected region R0 is consistent with the fault region R S The intersection area is greater than 0.8 times the fault area R S , the connected area is set as the real fault area to obtain the real fault area image, and a real fault area image dataset is constructed according to the real fault area image and the category of the real fault area image. Through the above operations, the fault area can be accurately located with high versatility, thereby greatly improving the accuracy and efficiency of fault detection.

[0118] (3) In order to solve the technical problem that the existing fault detection method combined with deep learning technology is prone to gradient vanishing and gradient exploding, this scheme inputs the real fault area image dataset into the R-FCN network model that introduces the residual network structure, and trains the model through the online difficult example mining method to obtain the fault detection model, thereby solving the technical problem that the existing fault detection method combined with deep learning technology is prone to gradient vanishing and gradient exploding, and improving the accuracy and efficiency of fault detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0119] Figure 1 A schematic diagram of the flow of a method for rapid detection of electrical cabinet faults based on computer vision provided by the present invention;

[0120] Figure 2 is a schematic flow chart of step S2;

[0121] Figure 3 is a flow chart of step S21;

[0122] Figure 4 is a flow chart of step S213;

[0123] Figure 5 is a flow chart of step S22;

[0124] Figure 6 is a flow chart of step S222;

[0125] Figure 7 is a schematic flow chart of step S3;

[0126] Figure 8 is a schematic diagram of the process of step S32;

[0127] Fig. 9 is a schematic flow chart of step S4;

[0128] Fig.10 is a flow chart of step S42;

[0129] Fig.11 is a schematic flow chart of step S5;

[0130] Fig.12 is a schematic diagram of the process of step S8;

[0131] Fig.13 is a flow chart of step S87;

[0132] Fig.14 This is a schematic diagram of the structure of the computer vision-based electric cabinet fault rapid detection device provided by the present invention.

[0133] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0134] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0135] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0136] Example 1, see Figure 1 The present invention provides a method for rapid detection of electric cabinet faults based on computer vision, comprising:

[0137] Step S1: image acquisition and construction of an image data set, specifically, acquisition of various types of infrared images of the interior of a distribution cabinet, and construction of an infrared image data set of the interior of a distribution cabinet using the infrared images of the interior of a distribution cabinet and the categories of the infrared images of the interior of a distribution cabinet;

[0138] Step S2: noise separation, specifically, separating high-frequency stripe noise and low-frequency thermal radiation noise from the infrared image inside the power distribution cabinet in the infrared image data set inside the power distribution cabinet, to obtain an infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed;

[0139] Step S3: marking the fault area, specifically, converting the infrared image from which high-frequency stripe noise and low-frequency thermal radiation noise are removed into an HSV color space image, performing clustering and normalization operations on the HSV color space image, and marking the pixels that meet the constraint conditions as the pixels of the fault area, thereby obtaining an HSV color space fault image;

[0140] Step S4: Determine the optimal threshold, specifically, determine the optimal threshold T of the V channel of the fault image in the HSV color space by using the OTSU algorithm M ;

[0141] Step S5: Determine the fault area, specifically, according to the optimal threshold T MPerform binarization to obtain a binary V channel image, perform morphological closing operation on the binary V channel image, and obtain the fault area R S ;

[0142] Step S6: connected region segmentation, specifically, segmenting the binary V channel image into multiple connected regions R0 by a connected component analysis algorithm;

[0143] Step S7: Determine the real fault area, specifically, traverse each connected area R0, if the connected area R0 is S The intersection area is greater than 0.8 times the fault area R S , then the connected area is set as the real fault area to obtain the real fault area image, and a real fault area image dataset is constructed according to the real fault area image and the category of the real fault area image;

[0144] Step S8: Model training, specifically, inputting the real fault area image dataset into the R-FCN network model with the residual network structure for training to obtain a fault detection model;

[0145] The R-FCN network model introducing the residual network structure includes a pre-trained convolutional neural network, a full convolutional network, a residual network structure, a ROI pooling layer, a classification layer and a regression layer;

[0146] The pre-trained convolutional neural network extracts a feature map of an input real fault area image dataset and sends the feature map to a full convolutional network;

[0147] The fully convolutional network converts the feature map into a dense category score map and a position regression map by introducing a residual network structure, sends the dense category score map to the classification layer, and sends the position regression map to the regression layer;

[0148] The fully convolutional network generates several ROIs by sliding a fixed-size window on the feature map, where the ROI is a region of interest;

[0149] The ROI pooling layer maps the ROI to obtain a pooled ROI feature map, and sends the pooled ROI feature map to the parallel classification layer and regression layer;

[0150] The classification layer and the regression layer are both fully connected layers, and the classification layer outputs the probability distribution and true label of each ROI belonging to different categories;

[0151] The regression layer outputs the bounding box coordinate adjustment and true label of each ROI;

[0152] Step S9: Fault detection, specifically, performing fault detection through a fault detection model to obtain the fault type, fault level and fault location;

[0153] In the above operation, due to the wide variety of components, parts and equipment inside the distribution cabinet, the existing fault detection methods generally have technical problems of difficulty in accurately locating the fault area and low versatility. This solution converts the infrared image with high-frequency stripe noise and low-frequency thermal radiation noise removed into an HSV color space image, performs clustering and normalization operations on the HSV color space image, and marks the pixels that meet the constraints as pixels in the fault area to obtain the HSV color space fault image. The optimal threshold T of the V channel of the HSV color space fault image is determined by the OTSU algorithm. M , and according to the optimal threshold T M Perform binarization to obtain a binary V channel image, perform morphological closing operation on the binary V channel image, and obtain the fault area R S , the binary V channel image is divided into multiple connected regions R0 by the connected component analysis algorithm, and each connected region R0 is traversed. If the connected region R0 is consistent with the fault region R S The intersection area is greater than 0.8 times the fault area R S , the connected area is set as the real fault area to obtain the real fault area image, and a real fault area image dataset is constructed according to the real fault area image and the category of the real fault area image. Through the above operations, the fault area can be accurately located with high versatility, thereby greatly improving the accuracy and efficiency of fault detection.

[0154] Embodiment 2, this embodiment is based on the above embodiment, see Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 In step S2, the noise separation step includes:

[0155] Step S21: Separating high-frequency stripe noise;

[0156] Step S22: Separating low-frequency thermal radiation noise;

[0157] In step S21, the step of separating high-frequency stripe noise includes:

[0158] Step S211: performing a two-dimensional discrete wavelet transform on the infrared image inside the power distribution cabinet, specifically, implementing a two-dimensional discrete wavelet transform through a digital filter and a downsampler, and obtaining an approximate image and a detailed image containing low-frequency components and high-frequency components through multiple iterations;

[0159] Step S212: Calculate the wavelet coefficients, specifically, calculate the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component, and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform according to the approximate image and the detailed image containing the low-frequency component and the high-frequency component, and the calculation formula of the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component, and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform is:

[0160]

[0161]

[0162]

[0163] In the formula, and They represent the wavelet coefficients of the vertical component, the horizontal component, and the diagonal component in the j-th scale space of the wavelet transform, respectively. M represents the number of rows of the image, N represents the number of columns of the image, m represents the row index of the image, and n represents the column index of the image. Indicates the summation of all pixels in the image. Represents the approximate image of the previous scale space of the j-th scale space, and They represent the wavelet function of the vertical component, the wavelet function of the horizontal component and the wavelet function of the diagonal component in the j-th scale space of the wavelet transform respectively;

[0164] Step S213: separating the high-frequency stripe noise according to the wavelet coefficients, specifically, using a cluster analysis algorithm to separate the high-frequency stripe noise according to the wavelet coefficients of the vertical component, the horizontal component and the diagonal component in each scale space of the wavelet transform, to obtain an infrared image with the high-frequency stripe noise removed;

[0165] In step S213, the step of separating high-frequency stripe noise according to wavelet coefficients includes:

[0166] Step S2131: clustering the wavelet coefficients of the vertical component, the horizontal component and the diagonal component in each scale space of the wavelet transform, classifying similar wavelet coefficients into the same category, and obtaining a clustering result;

[0167] Step S2132: extracting the wavelet coefficients belonging to the high-frequency stripe noise according to the clustering result, and obtaining a high-frequency stripe noise image;

[0168] Step S2133: performing a subtraction operation on the infrared image inside the power distribution cabinet and the high-frequency stripe noise image to obtain an infrared image with the high-frequency stripe noise removed;

[0169] In step S22, the step of separating low-frequency thermal radiation noise includes:

[0170] Step S221: performing wavelet domain transformation on the infrared image after removing high-frequency stripe noise to obtain a low-frequency component;

[0171] Step S222: Approximately estimate the low-frequency component using a Bezier surface fitting algorithm to obtain a Bezier surface

[0172] Step S223: Introduce the noise intensity coefficient λ and calculate the noise intensity coefficient according to the Bezier surface. A correction model is constructed, and the formula of the correction model is:

[0173]

[0174] In the formula, b represents the corrected image, λ represents the noise intensity coefficient, and × represents the multiplication operation. represents a Bezier surface;

[0175] Step S224: Calculate the noise intensity coefficient λ. Specifically, determine the noise intensity coefficient λ by minimizing the energy function. The calculation formula of the noise intensity coefficient λ is:

[0176] λ = argmin[J(λ)];

[0177] J(λ)={std[mean(I-λb)]-grad(I-λb)};

[0178] Where, λ represents the noise intensity coefficient, J(λ) represents the energy function of λ, argmin[J(λ)] represents the energy function that minimizes λ, b represents the corrected image, mean represents the calculation of the mean value of each column of the image to obtain a one-dimensional array, std represents the calculation of the standard deviation of the one-dimensional array, grad represents the calculation of the gradient of the image, and I represents the infrared image with high-frequency stripe noise removed;

[0179] Step S225: in the range of [0,1], a direct search method with a step size of 0.1 is used to obtain the optimal noise intensity coefficient λ;

[0180] Step S226: Substituting the optimal noise intensity coefficient λ into the correction model to remove the low-frequency thermal radiation noise, thereby obtaining an infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed;

[0181] In step S222, the step of approximating the low-frequency component by using the Bezier surface fitting algorithm includes:

[0182] Step S2221: define the value of the Bezier surface, the formula for defining the value of the Bezier surface is:

[0183]

[0184]

[0185] Where, the parameters u and v represent the position coordinates of the Bezier surface in the u direction and in the v direction, respectively, and the value range of u and v is [0,1]. b(u,v) represents the value of the point with the position coordinates (u,v) on the Bezier surface. i,j represents the coordinates of the control points of the Bezier surface, where i and j represent the index of the control point of the Bezier surface in the u direction and the index in the v direction, respectively, m and n represent the degree of the Bezier surface in the u direction and the degree of the Bezier surface in the v direction, respectively, t represents the interpolation position of the Bernstein polynomial, B i,m (u) represents the Bernstein polynomial of degree m on the Bezier surface in the u direction, B j,n (v) represents the Bernstein polynomial of degree n on the Bezier surface in the v direction, represents the number of combinations of selecting i elements from n elements, B i,n (t) represents the Bernstein polynomial of degree n at the interpolation position t;

[0186] Step S2222: constructing a matrix equation according to a formula defining the value of the Bezier surface;

[0187] Step S2223: Solve the matrix equation by the least square method to determine the control point coordinates P of the Bezier surface and obtain the Bezier surface

[0188] In the above operation, the existing noise separation method usually only focuses on high-frequency stripe noise, while ignoring the technical problem of the influence of low-frequency thermal radiation noise on image quality, and the lack of reasonable prior information to guide the noise structure, and the technical problem that the algorithm runs for a long time and easily causes parameters to fall into local optimum or parameter divergence by directly solving the optimal estimate by constructing a loss function. This scheme implements two-dimensional discrete wavelet transform through digital filters and downsamplers, and obtains approximate images and detailed images containing low-frequency components and high-frequency components through multiple iterations. According to the approximate images and detailed images containing low-frequency components and high-frequency components, the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform are calculated. According to the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform, a cluster analysis algorithm is used to separate the high-frequency stripe noise to obtain an infrared image with the high-frequency stripe noise removed, and a low-frequency component is obtained by performing a wavelet domain transform on the infrared image with the high-frequency stripe noise removed, and the low-frequency component is approximately estimated by a Bezier surface fitting algorithm to obtain a Bezier surface Introduce the noise intensity coefficient λ and according to the Bezier surface A correction model is constructed, and the noise intensity coefficient λ is determined by minimizing the energy function. In the range of [0,1], a direct search method with a step size of 0.1 is used to obtain the optimal noise intensity coefficient λ. The optimal noise intensity coefficient λ is substituted into the correction model to remove the low-frequency thermal radiation noise, and an infrared image with high-frequency stripe noise and low-frequency thermal radiation noise removed is obtained. Through the above operation, the high-frequency stripe noise and the low-frequency thermal radiation noise can be effectively separated from the infrared image, and the quality of the infrared image is effectively improved, thereby solving the technical problem that the existing noise separation method usually only focuses on the high-frequency stripe noise and ignores the influence of the low-frequency thermal radiation noise on the image quality; at the same time, the above operation can simply and efficiently eliminate the high-frequency stripe noise and the low-frequency thermal radiation noise without the need for reasonable prior information to guide the noise structure, and does not need to directly solve the optimal estimate by constructing a loss function, thereby solving the technical problem that the algorithm has a long running time and is prone to parameters falling into local optimality or parameter divergence due to the lack of reasonable prior information to guide the noise structure and the direct solution of the optimal estimate by constructing a loss function. Therefore, this scheme is suitable for occasions with strict requirements on image quality and processing speed.

[0189] Embodiment 3, this embodiment is based on the above embodiment, see Figure 7 and Figure 8 In step S3, the step of marking the fault area includes:

[0190] Step S31: Convert the infrared image with high-frequency stripe noise and low-frequency thermal radiation noise removed into an HSV color space image;

[0191] Step S32: Perform clustering operation on the HSV color space image to obtain the clustered HSV color space image;

[0192] Step S33: Normalize the clustered HSV color space image to ensure that the value of the H channel of the clustered HSV color space image remains within the range of [0, 360], and update the value of the H channel of the clustered HSV color space image to obtain the normalized HSV color space image;

[0193] Step S34: Set the constraint conditions: 10 < H < 150, 10 < S < 100, and 200 < V < 255, where H represents the value of the H channel of the HSV color space image, S represents the value of the S channel of the HSV color space image, and V represents the value of the V channel of the HSV color space image. Traverse all pixel points of the normalized HSV color space image. If the values of the H channel, S channel, and V channel of the pixel point all satisfy the constraint conditions, then set the values of the H channel, S channel, and V channel of the pixel point that satisfies the constraint conditions to 0, and mark the pixel point that satisfies the constraint conditions as a fault area pixel point to obtain the HSV color space fault image;

[0194] In step S32, the step of performing clustering operation on the HSV color space image includes:

[0195] Step S321: Initialize the clustering center points of the HSV color space image so that each clustering center point is evenly distributed in the HSV color space image according to the distance S;

[0196] Step S322: Calculate the gradient of the 3×3 neighborhood around each clustering center point, and select the pixel point with the minimum gradient as the new clustering center point within the 3×3 neighborhood;

[0197] Step S323: Calculate the distance metric D from each pixel point to each clustering center point, and assign the pixel point to the clustering center point with the minimum distance metric D.

[0198] Example 4, this example is based on the above example, refer to Fig. 9 and Fig.10 , in step S4, the step of determining the optimal threshold includes:

[0199] Step S41: Count the number of pixels and calculate the probability distribution, specifically, count the number of pixels of each gray level of the V channel of the HSV color space fault image, and calculate the probability distribution of each gray level in the V channel of the HSV color space fault image. The calculation formula for the probability distribution of each gray level in the V channel of the HSV color space fault image is:

[0200]

[0201] In the formula, p i represents the probability distribution of the i-th gray level in the V channel of the fault image in the HSV color space, N i represents the number of pixels of the i-th gray level, and n represents the number of pixels in the V channel of the fault image in the HSV color space;

[0202] Step S42: Calculate the optimal threshold T M ;

[0203] In step S42, the optimal threshold T is calculated. M The steps include:

[0204] Step S421: presetting the maximum value of the inter-class variance to 0, and setting the inter-class variance threshold T corresponding to the maximum value of the inter-class variance;

[0205] Step S422: Calculate the average gray value of the V channel of the fault image in the HSV color space;

[0206] Step S423: Calculate the probability distribution of the foreground and the background at each gray level. The calculation formula for the probability distribution of the foreground and the background at each gray level is:

[0207]

[0208] p1(T)=1-p0(T);

[0209] In the formula, p0(T) represents the probability distribution of the foreground at each gray level, p1(T) represents the probability distribution of the background at each gray level, T represents the inter-class variance threshold corresponding to the maximum inter-class variance, and N i represents the number of pixels of the i-th gray level, N represents the number of pixels in the V channel of the fault image in the HSV color space, and i represents the index of the gray level;

[0210] Step S424: Calculate the average grayscale value of the foreground and the average grayscale value of the background at each grayscale level. The calculation formula for the average grayscale value of the foreground and the average grayscale value of the background at each grayscale level is:

[0211]

[0212]

[0213] Where μ0(T) represents the average gray value of the foreground at each gray level, μ1(T) represents the average gray value of the background at each gray level, m represents the average gray value of the V channel of the fault image in the HSV color space, p0(T) represents the probability distribution of the foreground at each gray level, p1(T) represents the probability distribution of the background at each gray level, T represents the inter-class variance threshold, and N i represents the number of pixels of the i-th gray level, N represents the number of pixels in the fault image in the HSV color space, and i represents the index of the gray level;

[0214] Step S425: Calculate the inter-class variance. If the inter-class variance is greater than the maximum inter-class variance, update the maximum inter-class variance and the inter-class variance threshold T. The calculation formula of the inter-class variance is:

[0215]

[0216] In the formula, represents the inter-class variance, m represents the average gray value of the fault image in the HSV color space, p0(T) represents the probability distribution of the foreground at each gray level, p1(T) represents the probability distribution of the background at each gray level, T represents the inter-class variance threshold, μ0(T) represents the average gray value of the foreground at each gray level, and μ1(T) represents the average gray value of the background at each gray level;

[0217] Step S426: Set the inter-class variance threshold T corresponding to the maximum inter-class variance value as the optimal threshold T M .

[0218] Embodiment 5, this embodiment is based on the above embodiment, see Fig.11 In step S5, the step of determining the fault area includes:

[0219] Step S51: According to the optimal threshold T M Binarizing the V channel of the HSV color space fault image to obtain a binary V channel image, wherein the binary V channel image contains pixel points in the fault area;

[0220] Step S52: Connect the pixels of the fault region of the binary V channel image by morphological closing operation to obtain the fault region R S .

[0221] Embodiment 6, this embodiment is based on the above embodiment, see Fig.12 and Fig.13 In step S8, the model training step includes:

[0222] Step S81: extracting a feature map of an input real fault area image dataset through a pre-trained convolutional neural network;

[0223] Step S82: introducing a residual network structure into the full convolutional network, and converting the feature map into a dense category score map and a position regression map through the full convolutional network with the residual network structure;

[0224] Step S83: Generate several ROIs by sliding a fixed-size window on the feature map through the full convolutional network;

[0225] Step S84: Mapping the ROI through the ROI pooling layer to obtain a pooled ROI feature map;

[0226] Step S85: Output the probability distribution and true label of each ROI belonging to different categories through the classification layer, which is used to determine whether the ROI contains the fault area, and output the bounding box coordinate adjustment and true label of each ROI through the regression layer, which is used to accurately locate the position of the fault area;

[0227] Step S86: Perform model training by an online hard example mining method according to the probability distribution and true label of each ROI belonging to different categories output by the classification layer, and the bounding box coordinate adjustment and true label of each ROI output by the regression layer;

[0228] Step S87: Repeat steps S81 to S87 until a convergence condition is reached to obtain a fault detection model;

[0229] In step S87, the step of performing model training by using the online hard example mining method includes:

[0230] Step S871: During each iteration of training, the classification loss of each ROI is calculated according to the probability distribution of each ROI belonging to different categories and the true label output by the classification layer;

[0231] Step S872: pre-set a classification threshold, and set the ROI with a classification loss greater than the classification threshold as a ROI hard case;

[0232] Step S873: input the ROI difficult examples and the true labels corresponding to the ROI difficult examples into the regression layer, and calculate the regression loss;

[0233] Step S874: adding the classification loss of the ROI difficult example to the classification loss regression loss of the ROI difficult example to obtain the total loss of the ROI difficult example, and updating the parameters of the model through the optimization algorithm to minimize the total loss of the ROI difficult example;

[0234] In the above operation, in order to solve the technical problem that the existing fault detection method combined with deep learning technology is prone to gradient vanishing and gradient exploding, this solution inputs the real fault area image dataset into the R-FCN network model that introduces the residual network structure, and trains the model through the online difficult example mining method to obtain the fault detection model, thereby solving the technical problem that the existing fault detection method combined with deep learning technology is prone to gradient vanishing and gradient exploding, and improving the accuracy and efficiency of fault detection.

[0235] Embodiment 7, this embodiment is based on the above embodiment, see Fig.14 The computer vision-based electric cabinet fault rapid detection device provided by the present invention comprises an image acquisition module, a noise separation module, a fault area marking module, an optimal threshold determination module, a fault area determination module, a connected area segmentation module, a real fault area determination module, a model training module and a fault detection module;

[0236] The image acquisition module is used for image acquisition and constructing an image data set, specifically, acquiring various types of infrared images of the inside of the distribution cabinet, and constructing an infrared image data set of the inside of the distribution cabinet using the infrared images of the inside of the distribution cabinet and the categories of the infrared images of the inside of the distribution cabinet, and sending the infrared image data set of the inside of the distribution cabinet to the noise separation module;

[0237] The noise separation module is used for noise separation, specifically, separating high-frequency stripe noise and low-frequency thermal radiation noise from the infrared image inside the power distribution cabinet in the infrared image data set inside the power distribution cabinet, obtaining an infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed, and sending the infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed to the fault area marking module;

[0238] The fault area marking module is used for fault area marking, specifically, converting the infrared image from which high-frequency stripe noise and low-frequency thermal radiation noise are removed into an HSV color space image, performing clustering and normalization operations on the HSV color space image, marking pixels that meet the constraint conditions as pixels of the fault area, obtaining an HSV color space fault image, and sending the HSV color space fault image to the optimal threshold determination module;

[0239] The optimal threshold determination module is used to determine the optimal threshold, specifically, to determine the optimal threshold T of the V channel of the fault image in the HSV color space by using the OTSU algorithm. M , and the optimal threshold T M Send to the fault area determination module;

[0240] The fault area determination module is used to determine the fault area, specifically, according to the optimal threshold T MPerform binarization to obtain a binary V channel image, perform morphological closing operation on the binary V channel image, and obtain the fault area R S , and send the binary V channel image to the connected region segmentation module, and segment the fault region R S Send to the real fault area determination module;

[0241] The connected region segmentation module is used for connected region segmentation, specifically, segmenting the binary V channel image into multiple connected regions R0 through a connected component analysis algorithm, and sending the multiple connected regions R0 to the real fault region determination module;

[0242] The real fault area determination module is used to determine the real fault area. Specifically, each connected area R0 is traversed. If the connected area R0 is consistent with the fault area R S The intersection area is greater than 0.8 times the fault area R S , the connected area is set as the real fault area to obtain the real fault area image, a real fault area image dataset is constructed according to the real fault area image and the category of the real fault area image, and the real fault area image dataset is sent to the model training module;

[0243] The model training module is used for model training, specifically, inputting a real fault area image dataset into an R-FCN network model that introduces a residual network structure for training to obtain a fault detection model, and sending the fault detection model to the fault detection module;

[0244] The fault detection module is used for fault detection, specifically, performing fault detection through a fault detection model to obtain the fault type, fault level and fault location.

[0245] Embodiment 8: This embodiment is based on the above embodiment. The computer vision-based electric cabinet fault rapid detection system provided by the present invention includes a memory and a processor, and the processor executes a computer program stored in the memory.

[0246] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0247] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0248] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A method for rapid detection of electrical cabinet faults based on computer vision, characterized in that: include: Step S1: image acquisition and construction of an image data set, specifically, acquisition of various types of infrared images of the interior of a distribution cabinet, and construction of an infrared image data set of the interior of a distribution cabinet using the infrared images of the interior of a distribution cabinet and the categories of the infrared images of the interior of a distribution cabinet; Step S2: noise separation, specifically, separating high-frequency stripe noise and low-frequency thermal radiation noise from the infrared image inside the power distribution cabinet in the infrared image data set inside the power distribution cabinet, to obtain an infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed; Step S3: marking the fault area, specifically, converting the infrared image from which high-frequency stripe noise and low-frequency thermal radiation noise are removed into an HSV color space image, performing clustering and normalization operations on the HSV color space image, and marking the pixels that meet the constraint conditions as the pixels of the fault area, thereby obtaining an HSV color space fault image; Step S4: Determine the optimal threshold, specifically, determine the optimal threshold of the V channel of the fault image in the HSV color space by using the OTSU algorithm ; Step S5: Determine the fault area, specifically, according to the optimal threshold Perform binarization to obtain a binary V channel image, perform morphological closing operation on the binary V channel image to obtain the fault area ; Step S6: Connected region segmentation, specifically, segmenting the binary V channel image into multiple connected regions using a connected component analysis algorithm ; Step S7: Determine the real fault area, specifically, traverse each connected area , if the connected area With fault area The intersection area is greater than 0.8 times the fault area , then the connected area is set as the real fault area to obtain the real fault area image, and a real fault area image dataset is constructed according to the real fault area image and the category of the real fault area image; Step S8: model training, specifically, inputting the real fault area image dataset into the R-FCN network model with the residual network structure for training to obtain a fault detection model; The R-FCN network model introducing the residual network structure includes a pre-trained convolutional neural network, a full convolutional network, a residual network structure, a ROI pooling layer, a classification layer and a regression layer; The pre-trained convolutional neural network extracts a feature map of an input real fault area image dataset and sends the feature map to a full convolutional network; The fully convolutional network converts the feature map into a dense category score map and a position regression map by introducing a residual network structure, sends the dense category score map to the classification layer, and sends the position regression map to the regression layer; The fully convolutional network generates several ROIs by sliding a fixed-size window on the feature map, where the ROI is a region of interest; The ROI pooling layer maps the ROI to obtain a pooled ROI feature map, and sends the pooled ROI feature map to the parallel classification layer and regression layer; The classification layer and the regression layer are both fully connected layers, and the classification layer outputs the probability distribution and true label of each ROI belonging to different categories; The regression layer outputs the bounding box coordinate adjustment and the true label of each ROI; Step S9: Fault detection, specifically, performing fault detection through a fault detection model to obtain the fault type, fault level and fault location; In step S2, the noise separation step includes: Step S21: Separating high-frequency stripe noise; Step S22: Separating low-frequency thermal radiation noise; In step S21, the step of separating high-frequency stripe noise includes: Step S211: performing a two-dimensional discrete wavelet transform on the infrared image inside the power distribution cabinet, specifically, implementing a two-dimensional discrete wavelet transform through a digital filter and a downsampler, and obtaining an approximate image and a detailed image containing low-frequency components and high-frequency components through multiple iterations; Step S212: Calculate the wavelet coefficients, specifically, calculate the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component, and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform according to the approximate image and the detailed image containing the low-frequency component and the high-frequency component, and the calculation formula of the wavelet coefficients of the vertical component, the wavelet coefficients of the horizontal component, and the wavelet coefficients of the diagonal component in each scale space of the wavelet transform is: ； ； ； In the formula, , and They represent the wavelet coefficients of the vertical component, the horizontal component, and the diagonal component in the j-th scale space of the wavelet transform, respectively. M represents the number of rows of the image, N represents the number of columns of the image, m represents the row index of the image, and n represents the column index of the image. Indicates the summation of all pixels in the image. Represents the approximate image of the previous scale space of the j-th scale space, , and They represent the wavelet function of the vertical component, the wavelet function of the horizontal component and the wavelet function of the diagonal component in the j-th scale space of the wavelet transform respectively; Step S213: separating the high-frequency stripe noise according to the wavelet coefficients, specifically, using a cluster analysis algorithm to separate the high-frequency stripe noise according to the wavelet coefficients of the vertical component, the horizontal component and the diagonal component in each scale space of the wavelet transform, to obtain an infrared image with the high-frequency stripe noise removed; In step S213, the step of separating high-frequency stripe noise according to wavelet coefficients includes: Step S2131: clustering the wavelet coefficients of the vertical component, the horizontal component and the diagonal component in each scale space of the wavelet transform, classifying similar wavelet coefficients into the same category, and obtaining a clustering result; Step S2132: extracting the wavelet coefficients belonging to the high-frequency stripe noise according to the clustering result, and obtaining a high-frequency stripe noise image; Step S2133: performing a subtraction operation on the infrared image inside the power distribution cabinet and the high-frequency stripe noise image to obtain an infrared image with the high-frequency stripe noise removed; In step S22, the step of separating low-frequency thermal radiation noise includes: Step S221: performing wavelet domain transformation on the infrared image after removing high-frequency stripe noise to obtain a low-frequency component; Step S222: Approximately estimate the low-frequency component using a Bezier surface fitting algorithm to obtain a Bezier surface ; Step S223: Introducing noise intensity coefficient , and according to the Bezier surface A correction model is constructed, and the formula of the correction model is: ； In the formula, b represents the corrected image, represents the noise intensity coefficient, represents the multiplication operation, represents a Bezier surface; Step S224: Calculate the noise intensity coefficient Specifically, the noise intensity coefficient is determined by minimizing the energy function , the noise intensity coefficient The calculation formula is: ； ； In the formula, represents the noise intensity coefficient, express The energy function of Minimize The energy function of , b represents the corrected image, mean represents the calculation of the average value of each column of the image to obtain a one-dimensional array, and std represents the calculation of the standard deviation of the one-dimensional array. represents the gradient of the calculated image, and I represents the infrared image with high-frequency stripe noise removed; Step S225: In the range, a direct search method with a step size of 0.1 is used to obtain the optimal noise intensity coefficient. ; Step S226: The optimal noise intensity coefficient Substitute it into the correction model to remove the low-frequency thermal radiation noise, and obtain an infrared image with high-frequency stripe noise and low-frequency thermal radiation noise removed.

2. The method for rapid detection of electrical cabinet faults based on computer vision according to claim 1 is characterized in that: In step S222, the step of approximating the low-frequency component by using the Bezier surface fitting algorithm includes: Step S2221: define the value of the Bezier surface, the formula for defining the value of the Bezier surface is: ； ， ； In the formula, the parameters u and v represent the position coordinates of the Bezier surface in the u direction and the v direction respectively, and the value ranges of u and v are , The position coordinates on the Bezier surface are The value of the point, Represents the coordinates of the control points of the Bezier surface, where i and j represent the index of the control point of the Bezier surface in the u direction and the index in the v direction, respectively, m and n represent the degree of the Bezier surface in the u direction and the degree of the Bezier surface in the v direction, respectively, and t represents the interpolation position of the Bernstein polynomial. The Bernstein polynomial of degree m representing the Bezier surface in the u direction, represents the Bernstein polynomial of degree n on the Bezier surface in the v direction, It represents the number of combinations of selecting i elements from n elements. represents the Bernstein polynomial of degree n at interpolation position t; Step S2222: constructing a matrix equation according to a formula defining the value of the Bezier surface; Step S2223: Solve the matrix equation by the least square method to determine the control point coordinates P of the Bezier surface and obtain the Bezier surface .

3. The method for rapid detection of electrical cabinet faults based on computer vision according to claim 1 is characterized in that: In step S3, the step of marking the fault area includes: Step S31: converting the infrared image from which high-frequency stripe noise and low-frequency thermal radiation noise are removed into an HSV color space image; Step S32: performing a clustering operation on the HSV color space image to obtain a clustered HSV color space image; Step S33: Normalize the clustered HSV color space image to ensure that the value of the H channel of the clustered HSV color space image remains within Within the range, the value of the H channel of the clustered HSV color space image is updated to obtain a normalized HSV color space image; Step S34: Setting constraints: , and , where H represents the value of the H channel of the HSV color space image, S represents the value of the S channel of the HSV color space image, and V represents the value of the V channel of the HSV color space image. All pixels of the normalized HSV color space image are traversed. If the values ​​of the H channel, S channel, and V channel of the pixel point all meet the constraint conditions, the values ​​of the H channel, S channel, and V channel of the pixel point that meets the constraint conditions are all set to 0, and the pixel point that meets the constraint conditions is marked as a pixel point in the fault area, and the HSV color space fault image is obtained; In step S32, the step of performing clustering operation on the HSV color space image includes: Step S321: Initialize the cluster center points of the HSV color space image so that each cluster center point is evenly distributed in the HSV color space image according to the distance S; Step S322: For each cluster center point The gradient is calculated in the neighborhood and the pixel with the smallest gradient is selected as New cluster centers in the neighborhood; Step S323: Calculate the distance metric D from each pixel point to each cluster center point, and assign the pixel point to the cluster center point with the smallest distance metric D.

4. The method for rapid detection of electrical cabinet faults based on computer vision according to claim 1 is characterized in that: In step S4, the step of determining the optimal threshold comprises: Step S41: Count the number of pixels and calculate the probability distribution, specifically, count the number of pixels of each gray level of the V channel of the HSV color space fault image, and calculate the probability distribution of each gray level in the V channel of the HSV color space fault image. The calculation formula for the probability distribution of each gray level in the V channel of the HSV color space fault image is: ； In the formula, represents the probability distribution of the i-th gray level in the V channel of the fault image in the HSV color space, Represents the number of pixels of the i-th gray level, Indicates the number of pixels in the V channel of the fault image in the HSV color space; Step S42: Calculate the optimal threshold ; In step S42, the optimal threshold is calculated The steps include: Step S421: presetting the maximum value of the inter-class variance to 0, and setting the inter-class variance threshold T corresponding to the maximum value of the inter-class variance; Step S422: Calculate the average gray value of the V channel of the fault image in the HSV color space; Step S423: Calculate the probability distribution of the foreground and the background at each gray level. The calculation formula for the probability distribution of the foreground and the background at each gray level is: ； ； In the formula, represents the probability distribution of the foreground at each gray level, represents the probability distribution of the background at each gray level, T represents the inter-class variance threshold corresponding to the maximum inter-class variance, Represents the number of pixels of the i-th gray level, Represents the number of pixels in the V channel of the fault image in the HSV color space, and i represents the index of the gray level; Step S424: Calculate the average grayscale value of the foreground and the average grayscale value of the background at each grayscale level. The calculation formula for the average grayscale value of the foreground and the average grayscale value of the background at each grayscale level is: ； ； In the formula, represents the average gray value of the foreground at each gray level, represents the average gray value of the background at each gray level, m represents the average gray value of the V channel of the fault image in the HSV color space, represents the probability distribution of the foreground at each gray level, represents the probability distribution of the background at each gray level, T represents the inter-class variance threshold, Represents the number of pixels of the i-th gray level, It represents the number of pixels in the fault image in the HSV color space, and i represents the index of the gray level; Step S425: Calculate the inter-class variance. If the inter-class variance is greater than the maximum inter-class variance, update the maximum inter-class variance and the inter-class variance threshold T. The calculation formula of the inter-class variance is: ； In the formula, represents the inter-class variance, m represents the average gray value of the fault image in the HSV color space, represents the probability distribution of the foreground at each gray level, represents the probability distribution of the background at each gray level, T represents the inter-class variance threshold, represents the average gray value of the foreground at each gray level, Represents the average gray value of the background at each gray level; Step S426: Set the inter-class variance threshold T corresponding to the maximum inter-class variance value as the optimal threshold .

5. The method for rapid detection of electrical cabinet faults based on computer vision according to claim 1 is characterized in that: In step S5, the step of determining the fault area includes: Step S51: According to the optimal threshold Binarizing the V channel of the HSV color space fault image to obtain a binary V channel image, wherein the binary V channel image contains pixel points in the fault area; Step S52: Connect the pixels of the fault area of ​​the binary V channel image by morphological closing operation to obtain the fault area .

6. The method for rapid detection of electric cabinet faults based on computer vision according to claim 1 is characterized in that: In step S8, the model training step includes: Step S81: extracting a feature map of an input real fault area image dataset through a pre-trained convolutional neural network; Step S82: introducing a residual network structure into the full convolutional network, and converting the feature map into a dense category score map and a position regression map through the full convolutional network with the residual network structure; Step S83: Generate several ROIs by sliding a fixed-size window on the feature map through the full convolutional network; Step S84: Mapping the ROI through the ROI pooling layer to obtain a pooled ROI feature map; Step S85: Output the probability distribution and true label of each ROI belonging to different categories through the classification layer, which is used to determine whether the ROI contains the fault area, and output the bounding box coordinate adjustment and true label of each ROI through the regression layer, which is used to accurately locate the position of the fault area; Step S86: Perform model training by an online hard example mining method according to the probability distribution and true label of each ROI belonging to different categories output by the classification layer, and the bounding box coordinate adjustment and true label of each ROI output by the regression layer; Step S87: Repeat steps S81 to S87 until convergence conditions are reached to obtain a fault detection model.

7. The method for rapid detection of electric cabinet faults based on computer vision according to claim 6 is characterized in that: In step S87, the step of performing model training by using the online hard example mining method includes: Step S871: During each iteration of training, the classification loss of each ROI is calculated according to the probability distribution of each ROI belonging to different categories and the true label output by the classification layer; Step S872: pre-set a classification threshold, and set the ROI with a classification loss greater than the classification threshold as a ROI hard case; Step S873: input the ROI difficult examples and the true labels corresponding to the ROI difficult examples into the regression layer, and calculate the regression loss; Step S874: Add the classification loss of the ROI difficult example to the classification loss regression loss of the ROI difficult example to obtain the total loss of the ROI difficult example, and update the parameters of the model through the optimization algorithm to minimize the total loss of the ROI difficult example.

8. A device for rapid detection of electric cabinet faults based on computer vision, used to implement a method for rapid detection of electric cabinet faults based on computer vision as claimed in any one of claims 1 to 7, characterized in that: It includes an image acquisition module, a noise separation module, a fault area marking module, an optimal threshold determination module, a fault area determination module, a connected area segmentation module, a real fault area determination module, a model training module and a fault detection module; The image acquisition module is used for image acquisition and constructing an image data set, specifically, acquiring various types of infrared images of the inside of the distribution cabinet, and constructing an infrared image data set of the inside of the distribution cabinet using the infrared images of the inside of the distribution cabinet and the categories of the infrared images of the inside of the distribution cabinet, and sending the infrared image data set of the inside of the distribution cabinet to the noise separation module; The noise separation module is used for noise separation, specifically, separating high-frequency stripe noise and low-frequency thermal radiation noise from the infrared image inside the power distribution cabinet in the infrared image data set inside the power distribution cabinet, obtaining an infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed, and sending the infrared image from which the high-frequency stripe noise and the low-frequency thermal radiation noise are removed to the fault area marking module; The fault area marking module is used for fault area marking, specifically, converting the infrared image from which high-frequency stripe noise and low-frequency thermal radiation noise are removed into an HSV color space image, performing clustering and normalization operations on the HSV color space image, marking pixels that meet the constraint conditions as pixels of the fault area, obtaining an HSV color space fault image, and sending the HSV color space fault image to the optimal threshold determination module; The optimal threshold determination module is used to determine the optimal threshold, specifically, to determine the optimal threshold of the V channel of the HSV color space fault image by using the OTSU algorithm , and the optimal threshold Send to the fault area determination module; The fault area determination module is used to determine the fault area, specifically, according to the optimal threshold Perform binarization to obtain a binary V channel image, perform morphological closing operation on the binary V channel image to obtain the fault area , and send the binary V channel image to the connected region segmentation module to segment the fault area Send to the real fault area determination module; The connected region segmentation module is used for connected region segmentation, specifically, segmenting the binary V channel image into multiple connected regions through a connected component analysis algorithm. , and will be sent to multiple connected areas Send to the real fault area determination module; The real fault area determination module is used to determine the real fault area, specifically, traverse each connected area , if the connected area With fault area The intersection area is greater than 0.8 times the fault area , the connected area is set as the real fault area to obtain the real fault area image, a real fault area image dataset is constructed according to the real fault area image and the category of the real fault area image, and the real fault area image dataset is sent to the model training module; The model training module is used for model training, specifically, inputting a real fault area image dataset into an R-FCN network model that introduces a residual network structure for training to obtain a fault detection model, and sending the fault detection model to the fault detection module; The fault detection module is used for fault detection, specifically, performing fault detection through a fault detection model to obtain the fault type, fault level and fault location.

9. A computer vision-based rapid detection system for electric cabinet faults, comprising a memory and a processor, characterized in that: The processor executes the computer program stored in the memory to implement the computer vision-based rapid detection method for electric cabinet faults as described in any one of claims 1 to 7.

Images