Power grid overhead line foreign matter identification method based on machine vision

Abstract

The invention discloses a power grid overhead line foreign matter identification method based on machine vision. According to the method, the power grid overhead line is collected based on machine vision, the collected power grid overhead line image is preprocessed, the preprocessed image is sent to the trained convolutional neural network model for prediction, and the bounding box of the target object and the category of the target object are obtained. Determining whether the target object has a threat based on the category of the target object; by combining technical links such as timed image acquisition, image preprocessing, convolutional neural network target detection, security threat judgment and the like, the problems of low efficiency, insufficient accuracy, poor real-time performance and the like of a traditional foreign matter recognition method are solved, and the efficient, accurate and real-time foreign matter recognition method is provided.

Description

Technical Field

[0001] The present application belongs to the technical field of power grids, and more specifically, to a method for identifying foreign objects in overhead power lines of a power grid based on machine vision. Background Art

[0002] Overhead lines are an important part of power transmission, and their safety is directly related to the stable operation of the power system. However, since overhead lines are usually exposed to the natural environment, they are easily affected by various external factors, such as branches, bird nests, kites and other foreign objects. These foreign objects may not only cause safety accidents such as short circuits and fires, but may also lead to serious consequences such as power outages. Therefore, timely identification and removal of foreign objects on overhead lines is crucial to ensure the safe operation of the power system.

[0003] Traditional methods for identifying foreign objects on overhead lines mainly rely on manual inspections and simple image processing techniques. These methods have the following limitations:

[0004] Manual inspection is inefficient: Manual inspection requires a lot of manpower and time, especially in vast power grid areas, where the inspection work is very arduous and difficult to fully cover.

[0005] Insufficient accuracy: Simple image processing technology is difficult to cope with complex natural environments, such as lighting changes, background interference, etc., and is prone to missed detections and false detections.

[0006] Poor real-time performance: Traditional methods are difficult to achieve real-time monitoring and rapid response, which may cause foreign objects to remain undetected for a long time, thereby increasing safety risks. Summary of the invention

[0007] The present invention provides a method for identifying foreign objects in overhead power lines of a power grid based on machine vision, which is intended to solve the technical problems of low efficiency, insufficient accuracy and poor real-time performance of current manual inspections.

[0008] The present invention provides a method for identifying foreign objects in overhead power lines of a power grid based on machine vision, comprising the following steps:

[0009] Collect overhead line images at regular intervals and pre-process the collected images;

[0010] Use convolutional neural networks to predict the bounding box of each object and the category of each object;

[0011] If the target object is identified as a foreign object, it is judged that the foreign object poses a security threat and an alarm is triggered immediately.

[0012] The present invention collects power grid overhead lines based on machine vision, pre-processes the collected power grid overhead line images, sends the pre-processed images to a trained convolutional neural network model for prediction, obtains the bounding box of the target object and the category of the target object, and determines whether the target object poses a threat based on the category of the target object; the present invention solves the problems of low efficiency, insufficient accuracy and poor real-time performance of traditional foreign object recognition methods by combining technical links such as timed image acquisition, image pre-processing, convolutional neural network target detection and security threat judgment, and provides an efficient, accurate and real-time foreign object recognition method.

[0013] Preferably, the pretreatment comprises:

[0014] Image denoising: bilateral filtering is applied to the acquired image:

[0015]

[0016] Where: W p represents the normalization coefficient; f s (||(x i ,y i )-(x,y)||) represents the spatial Gaussian kernel, which depends on the difference of pixel positions; f r (|I(x i ,y i )-I(x,y)|) represents the range Gaussian kernel, which depends on the difference of pixel values; G(x,y) represents the pixel value after filtering; Ω represents the filtering window range; I(x i ,y i ) indicates that the input image is at position (x i ,y i ) out of the pixel value;

[0017] Image enhancement: Use histogram equalization method to enhance image contrast;

[0018] Image correction: Use distortion correction algorithm to correct the image;

[0019] Image resizing: resize images to a uniform size;

[0020] Data augmentation: affine transformation and lighting adjustment of images:

[0021] I affine =A·I+b;

[0022] I′=αI affine +β;

[0023] Where: I affinerepresents the image after affine transformation; I represents the image after image size adjustment; b represents the translation vector; I′ represents the image after illumination adjustment; β represents the brightness offset; α represents the contrast coefficient;

[0024] Color space conversion:

[0025] I enhanced =(I′-mean(I′))×saturation factor +mean(I′);

[0026] Where: saturation factor represents the enhancement factor; mean(I′) represents the average value of image I′, which is used to center the image; I enhanced Represents the image after color space conversion;

[0027] Grayscale conversion: Convert the image after color space conversion to grayscale image, and perform pseudo color conversion on the grayscale image:

[0028] I pseudo =colormap(I gray );

[0029] Where: I pseudo I represents a pseudo-color image; gray Represents a grayscale image.

[0030] Preferably, the convolutional neural network includes an input layer, a convolution layer, a pooling layer, a fully connected layer, an output layer and a loss function;

[0031] The convolution layer includes three layers. The first layer uses 32 3*3 convolution kernels to perform convolution operations to extract low-level features; the second layer uses 64 3*3 convolution kernels; and the third layer uses 128 3*3 convolution kernels.

[0032] Each of the convolutional layers is followed by a 2*2 maximum pooling layer to reduce the size of the feature map;

[0033] The fully connected layer includes two layers. The first fully connected layer includes 256 neurons, and the activation function is ReLu; the second fully connected layer has 128 neurons, and the activation function is ReLu; ReLu, Rectified Linear Unit, is a common activation function;

[0034] The output layer includes position and size prediction and category prediction; the position and size prediction is used to predict the bounding box of each target object; the category prediction is used to predict the probability that the target belongs to a foreign object, an animal or other category.

[0035] Preferably, the loss function of the convolutional neural network is as follows:

[0036] Position and size loss:

[0037]

[0038] Where: N represents the number of objects in the image; (x i ,y i ) represents the coordinates of the center point of the actual bounding box; Represents the center point coordinates of the predicted bounding box; (w i ,h i ) represents the width and height of the actual bounding box; Represents the width and height of the predicted bounding box; represents the indicator function, which is 1 if the i-th grid contains the target object, otherwise it is 0;

[0039] Confidence loss:

[0040]

[0041] Where: Indicates the degree of execution of the forecast; represents the indicator function, which is 1 if the i-th grid does not contain the target object, otherwise it is 0;

[0042] Classification loss:

[0043]

[0044] Where: p i,c Represents the actual category label of the i-th target; Represents the predicted category probability of the i-th target;

[0045] Total loss function:

[0046] L=λ loc L loc +λ conf L conf +λ class L class ;

[0047] Where: loc represents the weight of position and size loss; λ conf Represents the weight of confidence loss; λ class Represents the weight of the classification loss.

[0048] Preferably, a Dropout layer is added after the convolutional layer and the fully connected layer, the Dropout layer represents a random inactivation layer; and L2 regularization is used; wherein a 50% Dropout layer is added after the fully connected layer. L2 regularization is used to prevent overfitting of the machine learning model.

[0049] Preferably, during the training process of the convolutional neural network model, the generalization ability of the convolutional neural network model is improved by adding inter-class confusion samples;

[0050] Collect non-threatening samples of similar-looking specimens and label the collected non-threatening samples as non-threatening foreign bodies;

[0051] Data enhancement is performed on the inter-class confusion samples, and the data-enhanced inter-class confusion samples are mixed with threat samples to form a training data set.

[0052] Preferably, the data enhancement includes:

[0053] Rotation: Randomly rotate the image in the angle range of [-15°, 15°];

[0054] Scaling: Randomly scale the image with a scaling ratio range of [0.8, 1.2];

[0055] Translation: Randomly translate the image, and the translation distance range is within 10% of the image size;

[0056] Lighting adjustment: Randomly adjust the brightness and contrast of the image. The brightness adjustment range is [-0.2, 0.2], and the contrast adjustment range is [0.8, 1.2].

[0057] The beneficial effects of the present invention include:

[0058] The present invention collects power grid overhead lines based on machine vision, pre-processes the collected power grid overhead line images, sends the pre-processed images to a trained convolutional neural network model for prediction, obtains the bounding box of the target object and the category of the target object, and determines whether the target object poses a threat based on the category of the target object; the present invention solves the problems of low efficiency, insufficient accuracy and poor real-time performance of traditional foreign object recognition methods by combining technical links such as timed image acquisition, image pre-processing, convolutional neural network target detection and security threat judgment, and provides an efficient, accurate and real-time foreign object recognition method. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0060] Figure 1 The overall system architecture diagram provided for the embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] See also Figure 1 As shown, the best embodiment of the present invention is further described;

[0063] A method for identifying foreign objects in overhead power lines of a power grid based on machine vision comprises the following steps:

[0064] Collect overhead line images at regular intervals and pre-process the collected images;

[0065] Use convolutional neural networks to predict the bounding box of each object and the category of each object;

[0066] If the target object is identified as a foreign object, it is judged that the foreign object poses a security threat and an alarm is triggered immediately.

[0067] The present invention collects power grid overhead lines based on machine vision, pre-processes the collected power grid overhead line images, sends the pre-processed images to a trained convolutional neural network model for prediction, obtains the bounding box of the target object and the category of the target object, and determines whether the target object poses a threat based on the category of the target object; the present invention solves the problems of low efficiency, insufficient accuracy and poor real-time performance of traditional foreign object recognition methods by combining technical links such as timed image acquisition, image pre-processing, convolutional neural network target detection and security threat judgment, and provides an efficient, accurate and real-time foreign object recognition method.

[0068] Preferably, the pretreatment comprises:

[0069] Image denoising: bilateral filtering is applied to the acquired image:

[0070]

[0071] Where: W p represents the normalization coefficient; fs (||(x i ,y i )-(x,y)||) represents the spatial Gaussian kernel, which depends on the difference of pixel positions; f r (|I(x i ,y i )-I(x,y)|) represents the range Gaussian kernel, which depends on the difference of pixel values; G(x,y) represents the pixel value after filtering; Ω represents the filtering window range; I(x i ,y i ) indicates that the input image is at position (x i ,y i ) out of the pixel value;

[0072] In this embodiment, for the power grid overhead line image, it is very important to retain edge information, so bilateral filtering is used to consider the differences in space and color and retain edge details.

[0073] Image enhancement: Use histogram equalization method to enhance image contrast;

[0074] Image correction: Use distortion correction algorithm to correct the image;

[0075] Image resizing: resize images to a uniform size;

[0076] Data augmentation: affine transformation and lighting adjustment of images:

[0077] I affine =A·I+b;

[0078] I′=αI affine +β;

[0079] Where: I affine represents the image after affine transformation; I represents the image after image size adjustment; b represents the translation vector; I′ represents the image after illumination adjustment; β represents the brightness offset; α represents the contrast coefficient;

[0080] In this embodiment, for overhead line images, possible deformation and illumination changes of foreign objects in the actual environment are taken into consideration, so data enhancement is used to make the image more robust in a variable actual environment and adapt to foreign object detection under different viewing angles and lighting conditions.

[0081] Color space conversion:

[0082] I enhanced =(I′-mean(I′))×saturation factor +mean(I′);

[0083] Where: saturationfactor represents the enhancement factor; mean(I′) represents the average value of image I′, which is used to center the image; I enhanced Represents the image after color space conversion;

[0084] Grayscale conversion: Convert the image after color space conversion to grayscale image, and perform pseudo color conversion on the grayscale image:

[0085] I pseudo =colormap(I gray );

[0086] Where: I pseudo I represents a pseudo-color image; gray Represents a grayscale image.

[0087] Since a variety of foreign body colors may appear in practical applications, color enhancement and pseudo-color conversion methods are introduced to represent different grayscale values ​​with different colors, improve visual differences, and better highlight the key information in the image.

[0088] Preferably, the convolutional neural network includes an input layer, a convolution layer, a pooling layer, a fully connected layer, an output layer and a loss function;

[0089] The convolution layer includes three layers. The first layer uses 32 3*3 convolution kernels to perform convolution operations to extract low-level features; the second layer uses 64 3*3 convolution kernels; and the third layer uses 128 3*3 convolution kernels.

[0090] Each of the convolutional layers is followed by a 2*2 maximum pooling layer to reduce the size of the feature map;

[0091] The fully connected layer includes two layers, the first fully connected layer includes 256 neurons, and the activation function is ReLu; the second fully connected layer has 128 neurons, and the activation function is ReLu;

[0092] The output layer includes position and size prediction and category prediction; the position and size prediction is used to predict the bounding box of each target object; the category prediction is used to predict the probability that the target belongs to a foreign object, an animal or other category.

[0093] Preferably, the loss function of the convolutional neural network is as follows:

[0094] Position and size loss:

[0095]

[0096] Where: N represents the number of objects in the image; (x i ,y i ) represents the coordinates of the center point of the actual bounding box; Represents the center point coordinates of the predicted bounding box; (w i ,h i ) represents the width and height of the actual bounding box; Represents the width and height of the predicted bounding box; represents the indicator function, which is 1 if the i-th grid contains the target object, otherwise it is 0;

[0097] Confidence loss:

[0098]

[0099] Where: Indicates the degree of execution of the forecast; represents the indicator function, which is 1 if the i-th grid does not contain the target object, otherwise it is 0;

[0100] Classification loss:

[0101]

[0102] Where: p i,c Represents the actual category label of the i-th target; Represents the predicted category probability of the i-th target;

[0103] Total loss function:

[0104] L=λ loc L loc +λ conf L conf +λ class L class ;

[0105] Where: loc represents the weight of position and size loss; λ conf Represents the weight of confidence loss; λ class Represents the weight of the classification loss.

[0106] In this embodiment, the total loss function combines position and size loss, confidence loss and classification loss, wherein the position and size loss ensures that the predicted bounding box can accurately cover the target by calculating the square difference between the center point coordinates and the width and height of the bounding box, thereby improving the accuracy of target detection; and for confidence loss, the model can better distinguish between areas containing targets and areas not containing targets, reducing false detections and missed detections, and also calculates losses for areas not containing targets, encouraging the model to output lower confidence in these areas, thereby effectively suppressing the impact of background noise; through class loss, the model can accurately predict the category of the target and improve the classification accuracy of foreign objects or animals.

[0107] Preferably, a Dropout layer is added after the convolutional layer and the fully connected layer; and L2 regularization is used; wherein a 50% Dropout layer is added after the fully connected layer.

[0108] Preferably, during the training process of the convolutional neural network model, the generalization ability of the convolutional neural network model is improved by adding inter-class confusion samples;

[0109] Collect non-threatening samples of similar-looking specimens and label the collected non-threatening samples as non-threatening foreign bodies;

[0110] Data enhancement is performed on the inter-class confusion samples, and the data-enhanced inter-class confusion samples, normal samples and threat samples are mixed to form a training data set.

[0111] In the training data set, inter-class confusion samples account for 20% of the total samples, normal samples account for 50% of the total samples, and threat samples account for 30% of the total samples.

[0112] In this embodiment, by introducing non-threatening samples with similar appearance, the model can better distinguish targets of similar appearance but different categories, improve the adaptability to complex scenarios in practical applications, and reduce the false detection rate.

[0113] Preferably, the data enhancement includes:

[0114] Rotation: Randomly rotate the image in the angle range of [-15°, 15°];

[0115] Scaling: Randomly scale the image with a scaling ratio range of [0.8, 1.2];

[0116] Translation: Randomly translate the image, and the translation distance range is within 10% of the image size;

[0117] Lighting adjustment: Randomly adjust the brightness and contrast of the image. The brightness adjustment range is [-0.2, 0.2], and the contrast adjustment range is [0.8, 1.2].

[0118] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for identifying foreign objects in overhead power lines based on machine vision, characterized in that: The following steps are involved: Collect overhead line images at regular intervals and pre-process the collected images; Use convolutional neural networks to predict the bounding box of each object and the category of each object; If the target object is identified as a foreign object, it is judged that the foreign object poses a security threat and an alarm is triggered immediately.

2. The method for identifying foreign objects in overhead power lines of a power grid based on machine vision according to claim 1, characterized in that: The pre-processing comprises: Image denoising: bilateral filtering is applied to the acquired image: Where: W p represents the normalization coefficient; f s (||(x i ,y i )-(x,y)||) represents the spatial Gaussian kernel, which depends on the difference of pixel positions; f r (|I(x i ,y i )-I(x,y)|) represents the range Gaussian kernel, which depends on the difference of pixel values; G(x,y) represents the pixel value after filtering; Ω represents the filtering window range; I(x i ,y i ) indicates that the input image is at position (x i ,y i ) out of the pixel value; Image enhancement: Use histogram equalization method to enhance image contrast; Image correction: Use distortion correction algorithm to correct the image; Image resizing: resize images to a uniform size; Data augmentation: affine transformation and lighting adjustment of images: I affine =A·I+b; I′=αI affine +b; Where: I affine represents the image after affine transformation; I represents the image after image size adjustment; b represents the translation vector; I′ represents the image after illumination adjustment; β represents the brightness offset; α represents the contrast coefficient.

3. The method for identifying foreign objects in overhead power lines based on machine vision according to claim 2 is characterized in that: The pre-processing further comprises: Color space conversion: I enhanced =(I′-mean(I′))×saturation factor +mean(I′); Where: saturation factor represents the enhancement factor; mean(I′) represents the average value of image I′, which is used to center the image; I enhanced Represents an image after color space conversion.

4. A method for identifying foreign objects in overhead power lines based on machine vision according to claim 2 or 3, characterized in that: The pre-processing comprises: Grayscale conversion: Convert the image after color space conversion to grayscale image, and perform pseudo color conversion on the grayscale image: I pseudo =colormap(I gray ); Where: I pseudo I represents a pseudo-color image; gray Represents a grayscale image.

5. The method for identifying foreign objects in overhead power lines of a power grid based on machine vision according to claim 1, characterized in that: The convolutional neural network includes an input layer, a convolution layer, a pooling layer, a fully connected layer, an output layer and a loss function; The convolution layer includes three layers. The first layer uses 32 3*3 convolution kernels to perform convolution operations to extract low-level features; the second layer uses 64 3*3 convolution kernels; and the third layer uses 128 3*3 convolution kernels. Each of the convolutional layers is followed by a 2*2 maximum pooling layer to reduce the size of the feature map; The fully connected layer includes two layers. The first fully connected layer includes 256 neurons, and the activation function is ReLu; the second fully connected layer has 128 neurons, and the activation function is ReLu.

6. The method for identifying foreign objects in overhead power lines of a power grid based on machine vision according to claim 5, characterized in that: The output layer includes position and size prediction and category prediction; the position and size prediction is used to predict the bounding box of each target object; the category prediction is used to predict the probability that the target belongs to a foreign object, an animal or other category.

7. The method for identifying foreign objects in overhead power lines of a power grid based on machine vision according to claim 5, characterized in that: The loss function of the convolutional neural network is as follows: Position and size loss: Where: N represents the number of objects in the image; (x i ,y i ) represents the coordinates of the center point of the actual bounding box; Represents the coordinates of the center point of the predicted bounding box; (w i ,h i ) represents the width and height of the actual bounding box; Represents the width and height of the predicted bounding box; represents the indicator function, which is 1 if the i-th grid contains the target object, otherwise it is 0; Confidence loss: Where: Indicates the degree of execution of the forecast; represents the indicator function, which is 1 if the i-th grid does not contain the target object, otherwise it is 0; Classification loss: Where: p i,c Represents the actual category label of the i-th target; Represents the predicted category probability of the i-th target; Total loss function: L=λ loc L loc +λ conf L conf +λ class L class ; Where: loc The weights representing the position and size losses; λ conf Represents the weight of confidence loss; λ class Represents the weight of the classification loss.

8. The method for identifying foreign objects in overhead power lines of a power grid based on machine vision according to claim 6, characterized in that: A Dropout layer is added after the convolutional layer and the fully connected layer; and L2 regularization is used; wherein a 50% Dropout layer is added after the fully connected layer.

9. The method for identifying foreign objects in overhead power lines of a power grid based on machine vision according to claim 1, characterized in that: During the training process of the convolutional neural network model, the generalization ability of the convolutional neural network model is improved by adding inter-class confusion samples; Collect non-threatening samples of similar-looking specimens and label the collected non-threatening samples as non-threatening foreign bodies; Data enhancement is performed on the inter-class confusion samples, and the data-enhanced inter-class confusion samples, normal samples and threat samples are mixed to form a training data set.

10. The method for identifying foreign objects in overhead power lines of a power grid based on machine vision according to claim 9, characterized in that: The data enhancement includes: Rotation: Randomly rotate the image in the angle range of [-15°, 15°]; Scaling: Randomly scale the image with a scaling ratio range of [0.8, 1.2]; Translation: Randomly translate the image, and the translation distance range is within 10% of the image size; Lighting adjustment: Randomly adjust the brightness and contrast of the image. The brightness adjustment range is [-0.2, 0.2], and the contrast adjustment range is [0.8, 1.2].

Images