A Lightning Strike Active Feature Recognition Method for Transmission Line Images

Abstract

The present invention discloses a method for active lightning strike feature recognition in transmission line images, comprising the following steps: converting an image from RGB to a YUV matrix using matrix multiplication and addition, extracting edge information from a brightness matrix using a Candy algorithm, and then extracting edge feature information from the brightness matrix using a 3*3 Gaussian kernel; performing a convolution operation on the brightness matrix Y; calculating the horizontal and vertical gradients of each pixel in the brightness matrix using a 3*3 window; traversing each pixel in the brightness matrix; and identifying the active lightning strike feature in the horizontal gradient matrix if the horizontal gradient value of the current pixel is greater than the gradient value of its horizontally adjacent point or its brightness value exceeds 0.8 times the maximum brightness value. The present invention does not require any data pre-labeling or model training, resulting in lower overall development costs; requires less computation, and can be deployed on terminal devices, reducing system complexity and communication costs; is highly interpretable, and the input-output relationship is more intuitive, enabling better guidance for operation and maintenance.

Description

Technical Field

[0001] The present invention relates to a method for identifying active features of lightning strikes, and in particular to a method for identifying active features of lightning strikes in transmission line images. Background Art

[0002] Transmission line online monitoring equipment has been widely installed on lines or towers in various regions and at various voltage levels to collect real-time data, including current, voltage, meteorological, image and other data. Among them, image-based online monitoring equipment has more intuitive output results and can clearly reflect the actual situation on site. For example, line damage caused by large mechanical equipment or abnormal activities near towers can be better presented, so that line operation and maintenance personnel can be notified in the first time to handle them and ensure the power supply reliability of the power grid. Among transmission line faults, tripping faults caused by lightning strikes account for the vast majority. Therefore, capturing pictures or videos of the moment when lightning flashes over and hits the line can greatly accelerate fault diagnosis and line maintenance, thereby proposing a method for active lightning feature recognition of transmission line images to ensure line operation stability.

[0003] Reference Figure 3 One solution in the prior art is to use machine learning or deep learning algorithms. The specific solution of this method is:

[0004] 1. Sample organization: Collect a large number of historical images, combine them with actual lightning tripping situations, and manually assign a label to each image (for example, "lightning strike" or "no lightning strike").

[0005] 2. Feature extraction: Convert all pixel values of each historical image sample into a one-dimensional vector x = [x1, x2, …, xn] as the input of the neural network model;

[0006] 3. Build a model: Establish a multi-layer neural network, including an input layer, a hidden layer, and an output layer. The input layer is related to the number of pixels in the image; the hidden layer is composed of multiple layers, each containing multiple neurons, and each neuron contains a linear function (i.e., y = wx + b, where x is the input, y is the output, w is the weight matrix, and b is the bias matrix) and an activation function (e.g., g(x) = max(0, x), which is used to control the output within a specific range to prevent parameter expansion from getting out of control); the output layer contains two nodes, a and b, which represent the labels of the algorithm output. If a>b, it means "there is a lightning strike", otherwise it means "there is no lightning strike";

[0007] 4. Training the model: By feeding a large number of training samples into the built model, the back propagation algorithm is used to continuously optimize the parameters in the model so that the model output labeling category is as close as possible to the manual classification category, and the trained model is saved;

[0008] 5. Model classification: After extracting features from new samples, input them into the trained model to obtain the labeling results of the sample in the algorithm.

[0009] However, the principle of the neural network algorithm in the above-mentioned lightning strike active feature recognition method determines that the parameters in the network do not have the ability to reason. On the contrary, it is essentially data-driven and must use massive amounts of data to continuously iterate the parameters to the most appropriate values. The neural network model often involves dozens or even hundreds of hidden layers, and each layer contains thousands of neurons. Therefore, the amount of calculation for each image is very large, which is difficult for the processor of general terminal devices to support. Therefore, most of them are uploaded to the cloud server before calculation. This increases the communication cost on the one hand and increases the system complexity and the burden on the cloud server on the other hand.

[0010] Deep learning is essentially a black box. The algorithm is driven by data rather than mechanisms, so it lacks explainability and the correlation between input and output is not intuitive enough. If identification errors are made, it is difficult to make targeted improvements unless the data samples are significantly upgraded and retrained. Therefore, there is room for improvement. Summary of the Invention

[0011] The present invention overcomes the deficiencies of the prior art and provides a method for identifying active lightning strike features in transmission line images.

[0012] To achieve the above object, the present invention adopts a technical solution: a method for identifying active lightning strike features in transmission line images, comprising the following steps:

[0013] Step 1: Use matrix multiplication and addition to convert the image from RGB to YUV matrix, take the brightness matrix to extract edge information, and then use the Candy algorithm to extract edge feature information of the brightness matrix; use a 3*3 Gaussian kernel to traverse the brightness matrix Y for convolution operation; use a 3*3 window to calculate the horizontal and vertical gradients of each pixel in the brightness matrix;

[0014] Traverse each pixel in the brightness matrix; in the horizontal gradient matrix, if the horizontal gradient value of the current pixel is greater than the gradient value of its horizontal adjacent point or its brightness value exceeds 0.8 times the brightness maximum, then retain the horizontal gradient value of the pixel; otherwise, set it to 0; in the vertical gradient matrix, if the vertical gradient value of the current pixel is greater than the gradient value of its vertical adjacent point or its brightness value exceeds 0.8 times the brightness maximum, then retain the vertical gradient value of the pixel; otherwise, set it to 0; combine the horizontal gradient matrix and the vertical gradient matrix, that is, only save the pixels with values in the horizontal gradient matrix or the vertical gradient matrix in the brightness matrix Y, thereby completing non-maximum suppression;

[0015] Traverse each pixel in the luminance matrix Y. If the horizontal gradient value or the vertical gradient value of the pixel is greater than the high threshold, it is a strong edge point, and its pixel value is retained; if the horizontal gradient value or the vertical gradient value is greater than the low threshold and less than the high threshold, it is a weak edge point. If there is a strong edge point around it, its pixel value is retained, otherwise it is set to 0; the pixel values of other pixels are all set to 0, thus completing the double-threshold detection.

[0016] Step 2: Set a 11*11 window to slide on the image matrix, with a horizontal step size of 1 and a vertical step size of 11. Calculate the mean value u within each window, record the position corresponding to the maximum mean value d, and calculate the standard deviation s of the abscissa; when a lightning current channel is formed, there is an optical response, and the flashover trajectory is tracked according to the luminance values of each pixel.

[0017] Step 3: Since the lightning flashover trajectory has the characteristics of high luminance and vertically downward, it is possible to judge whether it is a lightning flashover according to the luminance mean value u and the standard deviation s of its abscissa extracted from the trajectory obtained by the above sliding window; taking the maximum window mean value d as the benchmark above, search for pixels whose luminance values exceed 0.5 times of its maximum mean value around, and record the positions of the pixels whose luminance values meet the conditions, which are the lightning flashover trajectories.

[0018] If u > 0.5d and s < p, the output is "lightning strike", otherwise the output is "no lightning strike".

[0019] p is the set threshold of 1.5.

[0020] In a preferred embodiment of the present invention, rgb respectively represent red, green and blue, and yuv respectively represent luminance, hue and saturation.

[0021] In a preferred embodiment of the present invention, the yuv matrix calculation formula is:

[0022]

[0023] In a preferred embodiment of the present invention, the Gaussian kernel matrix expression is:

[0024]

[0025] In a preferred embodiment of the present invention, the horizontal axis gradient expression is: G x = sum(-x(i - 1, j - 1) + x(i + 1, j - 1) - 2x(i - 1, j) + 2x(i + 1, j) - x(i - 1, j + 1) + x(i + 1, j + 1)); where x(i, j) is the luminance matrix value of the current pixel, and i, j represent the horizontal and vertical coordinate positions of the pixel.

[0026] In a preferred embodiment of the present invention, the vertical axis gradient expression is: G y=sum(x(i-1,j-1)+2x(i,j-1)+x(i+1,j-1)-x(i-1,j+1)-2x(i,j+1)-x(i+1,j+1)); where x(i,j) is the current pixel brightness matrix value, and i and j represent the horizontal and vertical coordinate positions of the pixel.

[0027] In a preferred embodiment of the present invention, the mean value u represents the average brightness value of a pixel set in a lightning flashover trajectory, and its calculation formula is: Where N represents the number of all pixels in the sliding window; I(x i ,y i ) represents the brightness value of each pixel.

[0028] In a preferred embodiment of the present invention, the horizontal coordinate standard deviation s represents the degree of discreteness of pixel distribution in the horizontal direction of the lightning flashover trajectory. The calculation formula is based on the horizontal coordinates of pixels in the flashover trajectory whose brightness values exceed a certain threshold. Among them, x i Represents the horizontal coordinates x1, x2, ..., x of these pixels n ; represents the mean of the horizontal axis, The calculation formula is:

[0029] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0030] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0031] The present invention solves the defects existing in the background technology and has the following beneficial effects:

[0032] Based on the image features of lightning, the present invention performs computational operations on images at the pixel level and outputs an identification of whether the features are present, thereby distinguishing whether the image contains lightning strikes. It does not require any data pre-labeling or model training, resulting in lower overall development costs. It has a small computational load and can be deployed on terminal devices, reducing system complexity and communication costs. It is highly interpretable, with a more intuitive correlation between input and output, which can better guide operation and maintenance work. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments described in the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive efforts.

[0034] Figure 1 This is a schematic diagram of the recognition process of a method for active lightning strike feature recognition of a transmission line image proposed by the present invention;

[0035] Figure 2 This is a schematic diagram of the image preprocessing process of a method for active lightning strike feature recognition of transmission line images proposed by the present invention;

[0036] Figure 3 The figure is a schematic diagram of the prior art recognition process of a method for active lightning strike feature recognition of transmission line images proposed by the present invention. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0039] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the scope of protection of the present application. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0040] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on specific circumstances.

[0041] Online monitoring equipment for power transmission lines has been widely used, especially visualization products, including photos and videos. These products can very intuitively reflect the operating status of the line within the equipment's monitoring range, thereby better guiding line operation and maintenance work. However, since line distances can be as long as hundreds of kilometers and the number of cameras deployed along the line is large, they continuously generate massive amounts of image information. In order to quickly diagnose and repair faults, line operators need to filter out a very small number of images or video frames that can capture the flashover moment of lightning faults, which is a huge workload. Therefore, it is necessary to develop a lightning flashover recognition method that automatically searches for images containing lightning flashover moments based on the image information of transmission line visualization online monitoring equipment, thereby reducing the human burden.

[0042] However, there are a large number of such devices, which generate massive amounts of image data at all times. Operation and maintenance personnel need to identify them one by one. Only by finding the very few that actually have lightning flashover moments can further diagnosis and maintenance be meaningful. Therefore, it is necessary to design an algorithm to filter out images of transmission line sites that do not contain lightning flashover features, and only hand over images with lightning flashover features to relevant personnel for further identification. This can greatly reduce labor costs. Therefore, the following technical solutions are specially disclosed:

[0043] like Figure 1 and Figure 2 As shown, a method for active lightning strike feature recognition of transmission line images includes the following steps:

[0044] Use matrix multiplication and addition to convert the image from RGB to YUV matrix, where RGB represents red, green, and blue, and YUV represents brightness, hue, and saturation. The YUV matrix calculation formula is: The brightness matrix is used to extract edge information, and then the candy algorithm is used to extract edge feature information of the brightness matrix; a 3*3 Gaussian kernel is used to perform convolution operation on the brightness matrix Y; the Gaussian kernel matrix expression is: Calculate the horizontal and vertical gradients of each pixel in the brightness matrix using a 3*3 window;

[0045] The horizontal axis gradient expression is:

[0046] G x =sum(-x(i-1,j-1)+x(i+1,j-1)-2x(i-1,j)+2x(i+1,j)-x(i-1,j+1)+x(i+1,j+1))

[0047] ; Among them, x(i,j) is the current pixel brightness matrix value, i,j represent the horizontal and vertical coordinate positions of the pixel.

[0048] The vertical axis gradient expression is:

[0049] G y =sum(x(i-1,j-1)+2x(i,j-1)+x(i+1,j-1)-x(i-1,j+1)-2x(i,j+1)-x(i+1,j+1))

[0050] ; Among them, x(i,j) is the current pixel brightness matrix value, i,j represent the horizontal and vertical coordinate positions of the pixel.

[0051] The original non-maximum suppression is intended to refine the edge, but it can easily lead to edge discontinuity. In addition, since lightning flashover is narrow, in order to preserve the flashover trajectory, an optimization process is performed here:

[0052] Traverse each pixel in the brightness matrix; in the horizontal gradient matrix, if the horizontal gradient value of the current pixel is greater than the gradient value of its horizontal adjacent point or its brightness value exceeds 0.8 times the maximum brightness, then retain the horizontal gradient value of the pixel; otherwise, set it to 0; in the vertical gradient matrix, if the vertical gradient value of the current pixel is greater than the gradient value of its vertical adjacent point or its brightness value exceeds 0.8 times the maximum brightness, then retain the vertical gradient value of the pixel; otherwise, set it to 0;

[0053] Combine the horizontal gradient matrix and the vertical gradient matrix, that is, only save the pixels with values in the horizontal gradient matrix or the vertical gradient matrix in the brightness matrix Y, thereby completing non-maximum suppression;

[0054] Traverse each pixel in the brightness matrix Y. If the horizontal gradient value or vertical gradient value of the pixel is greater than the high threshold, it is a strong edge point and its pixel value is retained; if the horizontal gradient value or vertical gradient value is greater than the low threshold and less than the high threshold, it is a weak edge point. If there are strong edge points around it, its pixel value is retained, otherwise it is set to 0; all other pixel values are set to 0, thus completing the double threshold detection;

[0055] A 11*11 window is set to slide across the image matrix with a horizontal step size of 1 and a vertical step size of 11. The mean value u within each window is calculated, the position corresponding to the maximum mean value d is recorded, and the horizontal axis standard deviation s is calculated. When a lightning current channel is formed, accompanied by a light reaction, its flashover trajectory is tracked based on the brightness value of each pixel.

[0056] Since the lightning flashover trajectory has the characteristics of high brightness and vertical downward direction, it is possible to determine whether it is a lightning flashover according to the average brightness u and the standard deviation s of the abscissa in the trajectory extracted by the above sliding window. Based on the maximum value d of the window mean, search for pixels whose brightness values exceed 0.5 times the maximum mean around it, and record the positions of the pixels whose brightness values meet the conditions, which are the lightning flashover trajectories.

[0057] The mean u represents the average brightness value of the pixel set in the lightning flashover trajectory, and its calculation formula is: where N represents the number of all pixels in the sliding window; I(xi, yi) represents the brightness value of each pixel.

[0058] The standard deviation s of the abscissa represents the degree of dispersion of the pixel distribution of the lightning flashover trajectory in the horizontal direction. Its calculation is based on the abscissas of the pixels whose brightness values in the flashover trajectory exceed a certain threshold, and the calculation formula is: where xi represents the abscissas x1, x2,..., x of these pixels n ; represents the mean of the abscissas, The calculation formula of

[0059] If u > 0.5d and s < p, the output is "lightning strike", otherwise the output is "no lightning strike"; p is the set threshold 1.5.

[0060] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0061] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0062] In the picture of the lightning strike moment, there is often a white high-brightness line, and the direction is basically perpendicular to the ground. Therefore, this method specifically proposes a method that can extract this feature to identify the lightning flashover moment. In principle, this method can effectively avoid the problems of large data volume, large calculation amount, and poor interpretability existing in the prior art.

[0063] In order to verify the advantages of the present invention over the prior art, the following comparative experiments are carried out:

[0064] The data for each category was divided into 150 groups: 90 for training and 60 for testing. The lightning strike and no lightning strike labels were defined as 0 and 1, respectively. A bidirectional LSTM layer with 95 hidden units was specified, and the last element of the sequence was output. Finally, two classes were specified by including a fully connected layer of size 4, followed by a softmax layer and a classification layer. The LSTM solver was specified as adam, with a gradient threshold of 11 and a maximum number of epochs of 450. A total of 2800 iterations were performed. As training progressed, the accuracy continued to improve, and the loss function also tended to converge steadily, with a final training accuracy of 98.7%. Using the same configuration, the feature vectors were input into a BP neural network, a support vector machine (SVM), and a random forest (RF) algorithm for training and testing.

[0065] Table 1 shows the classification results of each method:

[0066]

[0067]

[0068] The results show that the LSTM classification method has a training set accuracy of 98.70% and a test set accuracy of 96.76%. Both results are significantly better than the other three methods. This proves that the LSTM classification method has higher accuracy and has a greater advantage than other classification algorithms, making significant progress.

[0069] The above description is based on the ideal embodiment of the present invention. Based on the above description, relevant personnel can make various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the content of the specification and must be determined according to the scope of the claims.

Claims

1. A method for identifying active lightning strike features in transmission line images, characterized in that: It includes the following steps: Step 1: Convert the picture from RGB to a YUV matrix using matrix multiplication and addition. Take the luminance matrix to extract edge information, and then use the Canny algorithm to extract the edge feature information of the luminance matrix. Perform a convolution operation by traversing with a 3×3 Gaussian kernel on the luminance matrix Y; calculate the horizontal axis gradient and vertical axis gradient of each pixel in the luminance matrix with a 3×3 window. Traverse each pixel in the luminance matrix; in the horizontal axis gradient matrix, if the horizontal axis gradient value of the current pixel is greater than the gradient value of its adjacent point in the horizontal direction or its luminance value exceeds 0.8 times the maximum luminance value, then retain the horizontal axis gradient value of this pixel, otherwise set it to 0; in the vertical axis gradient matrix, if the vertical axis gradient value of the current pixel is greater than the gradient value of its adjacent point in the vertical direction or its luminance value exceeds 0.8 times the maximum luminance value, then retain the vertical axis gradient value of this pixel, otherwise set it to 0; combine the horizontal axis gradient matrix and the vertical axis gradient matrix, that is, only save the pixels with values in either the horizontal axis gradient matrix or the vertical axis gradient matrix in the luminance matrix Y, thereby completing non-maximum suppression. Traverse each pixel in the luminance matrix Y. If the horizontal gradient value or vertical gradient value of this pixel is greater than the high threshold, it is a strong edge point, and retain its pixel value; if the horizontal gradient value or vertical gradient value is greater than the low threshold and less than the high threshold, it is a weak edge point. If there is a strong edge point around it, then retain its pixel value, otherwise set it to 0; set the values of other pixels to 0, thereby completing double-threshold detection. Step 2: Set a 11×11 window to slide on the image matrix, with a horizontal step size of 1 and a vertical step size of 11. Calculate the mean value u within each window, record the position corresponding to the maximum mean value d, and calculate the standard deviation s of the abscissa. The mean value u represents the average brightness value of the pixel set in the lightning flashover trajectory, and its calculation formula is: Where N represents the number of all pixels in the sliding window; I(x i ,y i ) represents the brightness value of each pixel; The standard deviation s of the horizontal axis represents the degree of discreteness of the pixel distribution in the horizontal direction of the lightning flashover trajectory. It is calculated based on the horizontal coordinates of the pixels whose brightness values exceed a certain threshold in the flashover trajectory. The calculation formula is: Among them, x i Represents the horizontal coordinates x1, x2, ..., x of these pixels n ; represents the mean of the horizontal axis, The calculation formula is: When a lightning current channel is formed, there is an optical reaction, and the flashover trajectory is tracked according to the luminance values of each pixel. Step 3: Since the lightning flashover trajectory has the characteristics of high luminance and vertical downward, it is possible to judge whether it is a lightning flashover based on the luminance mean value u and the standard deviation s of its abscissa extracted from the trajectory by the above sliding window; based on the maximum window mean value d above, search for pixels whose luminance values exceed 0.5 times its maximum mean value around, and record the positions of the pixels whose luminance values meet the conditions, which is the lightning flashover trajectory. If u > 0.5d and s < p, the output is "There is a lightning strike", otherwise the output is "There is no lightning strike"; p is the set threshold of 1.

5.

2. The method for identifying active lightning strike features in a transmission line image according to claim 1, characterized in that: The RGB respectively represent red, green, and blue, and the YUV respectively represent luminance, hue, and saturation.

3. The method for identifying active lightning strike features in a transmission line image according to claim 1, wherein: The calculation formula of the YUV matrix is:

4. The method for identifying active lightning strike features in a transmission line image according to claim 3, wherein: The Gaussian kernel matrix expression is:

5. The method for identifying active lightning strike features in a transmission line image according to claim 1, wherein: The expression of the horizontal axis gradient is: G x =sum(-x(i-1,j-1)+x(i+1,j-1)-2x(i-1,j)+2x(i+1,j)-x(i-1,j+1)+x(i+1,j+1)); where x(i,j) is the current pixel brightness matrix value, and i,j represent the horizontal and vertical coordinate positions of the pixel.

6. The method for identifying active lightning strike features in a transmission line image according to claim 5, characterized in that: The expression of the vertical axis gradient is: G y =sum(x(i-1,j-1)+2x(i,j-1)+x(i+1,j-1)-x(i-1,j+1)-2x(i,j+1)-x(i+1,j+1)); where x(i,j) is the current pixel brightness matrix value, and i and j represent the horizontal and vertical coordinate positions of the pixel.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the method described in any one of claims 1 to 6.

Images