A switch state recognition method based on optical flow and deep learning

Abstract

This invention proposes a disconnect switch status recognition method based on optical flow and deep learning, comprising a video acquisition module, a deep learning status recognition module, and an optical flow recognition module. The recognition method comprises the following steps: Step 1: The video acquisition module acquires disconnect switch video data, and the processor converts the disconnect switch video data into a continuous image sequence; Step 2: The first five frames of the image sequence are entered into the deep learning status recognition module for initial status recognition processing of the disconnect switch; Step 3: The deep learning status recognition module determines the initial status of the disconnect switch, issuing an alarm message if abnormal, and proceeding to the next step if normal; Step 4: The image sequence of the disconnect switch with a normal initial status is entered into the optical flow recognition module for processing, and tracking begins based on pre-set key points. If tracking is successful, the disconnect switch operating status information is output; if tracking fails, the last five frames of the image sequence are passed into the deep learning status recognition module for processing, and the disconnect switch operating status information is output.

Description

Technical Field

[0001] This invention relates to the field of disconnector status recognition technology, specifically a disconnector status recognition method based on optical flow and deep learning. Background Art

[0002] Disconnectors, as crucial equipment in substations, isolate voltage in circuits, ensuring the safety of maintenance personnel. Therefore, real-time monitoring of disconnector status is vital for stable substation operation and personnel safety. Monitoring and identifying disconnector status better protects the normal operation of power equipment and safeguards maintenance personnel. However, existing disconnector monitoring and identification methods or equipment are susceptible to factors such as ambient lighting, complex backgrounds, and inaccurate identification, resulting in high false detection rates. Furthermore, the wide variety of disconnectors necessitates a corresponding increase in detection equipment types, leading to higher construction costs and greater maintenance difficulty. To address this need, the applicant proposes a disconnector status identification method based on optical flow and deep learning. This method collects video data, analyzes the status of target disconnectors in the substation in real time, and issues alarms for abnormal states, thereby improving the reliability of substation operation and the safety of personnel. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes a disconnect switch status recognition method based on optical flow and deep learning. The method comprises a video acquisition module, a deep learning status recognition module, and an optical flow recognition module. The steps are as follows: Step 1: The video acquisition module acquires disconnect switch video data, and the processor converts the video data into a continuous image sequence. Step 2: The first five frames of the image sequence are fed into the deep learning status recognition module for initial status recognition processing of the disconnect switch. Step 3: The deep learning status recognition module determines the initial status of the disconnect switch; if abnormal, an alarm is issued; if normal, the image sequence proceeds to the next step. Step 4: The image sequence with a normal initial status of the disconnect switch is processed in the optical flow recognition module, and tracking begins based on pre-set key points. If tracking is successful, the disconnect switch operating status information is output; if tracking fails, the last five frames of the image sequence are fed into the deep learning status recognition module for processing, and the disconnect switch operating status information is output. This method offers high recognition accuracy and good security.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] A method for identifying the status of a disconnector based on optical flow and deep learning includes a video acquisition module, a deep learning status identification module, and an optical flow identification module. The method comprises the following steps:

[0006] Step 1: The video acquisition module acquires video data from the disconnect switch, and the processor converts the video data into a continuous image sequence.

[0007] Step 2: The first five frames of the image sequence are entered into the deep learning state recognition module for initial state recognition processing of the switch.

[0008] Step 3: The deep learning state recognition module determines the initial state of the disconnector. If there is an abnormality, an alarm message is issued. If the state is normal, the image sequence proceeds to the next step.

[0009] The algorithm flow corresponding to the deep learning state recognition module is described as follows:

[0010] First, data preprocessing: load training images and labels, scale and crop the images, and map and transform the labels accordingly;

[0011] Next, define the model: define the input layer, convolutional layer, pooling layer, and output layer;

[0012] Then the loss function is calculated: classification loss, confidence loss, coordinate loss, and total loss are calculated;

[0013] Next, train the model: define the optimizer and learning rate decay strategy, train on the training set, validate on the validation set, and save the model and weights;

[0014] Final prediction: Load the model and weights, preprocess the input image, run the model to get the output results, perform post-processing including NMS and parsing the output results, and display or save the results;

[0015] Step 4: The image sequence of the switch with normal initial state is entered into the optical flow recognition module for processing, and tracking begins based on the pre-set key points. If the trajectory of the key point movement is linear, the tracking is considered successful, and the switch operation status information is output. If the trajectory of the key point movement is non-linear, the tracking is considered unsuccessful, and the last five frames of the image sequence are passed into the deep learning state recognition module, which outputs the switch operation status information.

[0016] Furthermore, the switch status recognition method based on optical flow and deep learning includes a deep learning status recognition module. This module only performs switch status detection on key frames of the image sequence, resulting in low computational complexity.

[0017] Furthermore, the switch status recognition method based on optical flow and deep learning includes an optical flow recognition module, which is sensitive to angle changes and has high detection accuracy.

[0018] This invention provides a method for identifying the status of a disconnect switch based on optical flow and deep learning. The method comprises a video acquisition module, a deep learning status identification module, and an optical flow identification module. The steps are as follows: Step 1: The video acquisition module acquires video data from the disconnect switch, and the processor converts the video data into a continuous image sequence. Step 2: The first five frames of the image sequence are processed by the deep learning status identification module to identify the initial status of the disconnect switch. Step 3: The deep learning status identification module determines the initial status of the disconnect switch; if abnormal, an alarm is issued; if normal, the image sequence proceeds to the next step. Step 4: The image sequence with a normal initial status of the disconnect switch is processed by the optical flow identification module, and tracking begins based on pre-set key points. If tracking is successful, the disconnect switch operating status information is output; if tracking fails, the last five frames of the image sequence are passed to the deep learning status identification module for processing, and the disconnect switch operating status information is output. The advantages are:

[0019] 1. A switch status recognition method based on optical flow and deep learning is implemented by analyzing video data to achieve real-time monitoring of switch status;

[0020] 2. The disconnector status recognition method based on optical flow and deep learning is independently set up and is not limited by the disconnector equipment model;

[0021] 3. A disconnector status identification method based on optical flow and deep learning: The "double confirmation" method of disconnector status identification using optical flow and deep learning algorithms can improve the accuracy of disconnector status identification and better ensure power grid stability and staff safety. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the workflow of the present invention;

[0023] Figure 2 This is a schematic diagram of the installation of the device of the present invention. Detailed Implementation

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0025] like Figure 1 The image shows a method for identifying the status of a disconnector based on optical flow and deep learning. It includes a video acquisition module, a deep learning status identification module, and an optical flow identification module. The method is characterized by the following steps:

[0026] Step 1: The video acquisition module acquires video data from the disconnect switch. The processor converts the video data into a continuous image sequence and extracts key frame data from the video data for data analysis.

[0027] Step 2: The first five frames of the image sequence are fed into the deep learning state recognition module for initial state recognition processing of the switch; (The deep learning state recognition module uses the YOLOv4 model, which supports single-GPU training. The input image size is 416x416. The hidden layer uses CSPDarknet53 as the backbone network, SPP as an additional module of the Neck, PANet as the feature fusion module of the Neck network, and YOLOv3 as the head network.)

[0028] YOLOv4 Model Training: First, a deep learning detection model is trained using YOLOv4, which can be trained on all types of disconnectors. The training steps are as follows: Use a camera to collect images of disconnector equipment in substations, manually label the bounding rectangle of each disconnector in the images, and classify the disconnector status as: disconnector closed, disconnector open, and disconnector abnormal.

[0029] 1) Construct a YOLOv4 object detection network model with 3 categories to detect.

[0030] 2) Input the data labeled in step one into the model in step two, and train the model using stochastic gradient descent until the model converges.

[0031] The deep learning state recognition module reads keyframes from the image sequence. Keyframes are typically selected from the first five and last five frames, from which the result can be predicted, to avoid images with no detection results. Each keyframe is scaled to 416*416, and each RGB channel of the image is divided by 255. The normalized image is then fed into the YOLOv4 detection model, retaining all results with approximately a set confidence threshold. Based on the results of the first and last five frames, the initial and final states of the target switch are obtained.

[0032] The algorithm flow is described as follows:

[0033] First, data preprocessing: load training images and labels, scale and crop the images, and map and transform the labels accordingly.

[0034] Next, define the model: define the input layer, convolutional layer, pooling layer, and output layer.

[0035] Then the loss function is calculated: classification loss, confidence loss, coordinate loss, and total loss are calculated.

[0036] Next, train the model: define the optimizer and learning rate decay strategy, train on the training set, validate on the validation set, and save the model and weights.

[0037] Final prediction: Load the model and weights, preprocess the input image, run the model to obtain the output result, perform post-processing including NMS and parsing the output result, and display or save the result.

[0038] Step 3: The deep learning state recognition module determines the initial state of the disconnector. If there is an abnormality, an alarm message is issued. If the state is normal, the image sequence proceeds to the next step.

[0039] Step 4: The image sequence of the switch with normal initial state is entered into the optical flow recognition module for processing, and tracking begins based on the pre-set key points. If the trajectory of the key point movement is linear, the tracking is considered successful, and the switch operation status information is output. If the trajectory of the key point movement is non-linear, the tracking is considered unsuccessful, and the last five frames of the image sequence are passed into the deep learning state recognition module, which outputs the switch operation status information. Optical flow is a method that uses the temporal changes of pixels in an image sequence and the correlation between adjacent frames to find the correspondence between the previous frame and the current frame, thereby calculating the motion information of objects between adjacent frames. In this study, the LK optical flow algorithm, short for Lucas-Kanade optical flow algorithm, is used. Essentially, it solves the optical flow using the least squares method without iteration. The basic principle of this algorithm is as follows: A continuous sequence of video frames is processed. For each video sequence, a certain target detection method is used to detect possible foreground targets. If a foreground target appears in a frame, its representative key feature points are found. For any two subsequent adjacent video frames, the optimal position of the key feature points appearing in the previous frame in the current frame is found, thus obtaining the position coordinates of the foreground target in the current frame. This process is repeated iteratively to achieve target tracking.

[0040] The LK optical flow method relies on three fundamental assumptions: First, constant brightness. This means that the brightness of the same target remains unchanged as it moves between frames. This is a basic assumption of optical flow (which all variants of optical flow must satisfy) and is used to derive the fundamental equations of optical flow. Second, temporal continuity or "small motion." This means that changes in time do not cause drastic changes in the target's position, and the displacement between adjacent frames must be relatively small. Finally, spatial consistency. Adjacent points on the same surface in the scene exhibit similar motion, and their projections onto the plane are relatively close.

[0041] Based on the first two assumptions, we obtain the constraint equations for the image:

[0042] I(x,y,t)=I(x+δx,y+δy,t+δt)

[0043] Where I(x, y, t) is the brightness of the image at position (x, y) at time t.

[0044] Expanding the function I(x+δx, y+δy, t+δt) at (x, y, t) using Taylor's formula yields:

[0045]

[0046] Where R(x, y, t) is the higher-order remainder term of the Taylor formula, which is approximately 0.

[0047] Combining the two formulas above, we get the equivalent:

[0048]

[0049] in and Let u and v be the derivatives of the pixel along the x and y directions, respectively, and the velocity components along the x and y directions be denoted as u and v, respectively. The above formula can be simplified as:

[0050] I x u+I y v+I t =0

[0051] Since the above equations have two unknowns, u and v, they cannot be solved. Therefore, it is necessary to obtain some other equations based on the third assumption and solve them simultaneously.

[0052] Assuming the optical flow of an image is constant within a window of size m×m, then the following system of equations can be obtained:

[0053]

[0054] To solve the above over-constrained system, the least-two-level method can be used to minimize the above equations, which can be expressed in matrix form as follows:

[0055]

[0056] Record The least squares method yields the following:

[0057]

[0058] The final optical flow (velocity vector) obtained is:

[0059]

[0060] By combining information from several neighboring pixels, the LK optical flow method can typically eliminate ambiguity in the optical flow equation. Furthermore, compared to point-by-point calculation methods, the LK method is less sensitive to image noise.

[0061] The method for identifying the disconnector status based on optical flow and deep learning is shown. It sets up a deep learning status identification module, which only performs disconnector status detection on key frames of the image sequence, resulting in low computational complexity.

[0062] The switch status recognition method based on optical flow and deep learning is shown, which sets up an optical flow recognition module. The optical flow recognition module is sensitive to angle changes and has high detection accuracy.

[0063] like Figure 2 As shown, the core of the video acquisition module consists of a fixed camera mounted in front of the disconnect switch, a video analysis server, and network cables and equipment connecting the camera and the server. The camera can be a reused camera from the substation or a dedicated camera. The video analysis server does not refer to a specific type of computer; it can be a single computer or a distributed computer system, along with the necessary computer software based on that hardware. The camera acquires video stream data and transmits it to the video analysis server, which converts the video stream into a continuous sequence of images.

[0064] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A method for identifying the status of a disconnector based on optical flow and deep learning, comprising a video acquisition module, a deep learning status identification module, and an optical flow identification module, characterized in that: The steps of the switch status recognition method based on optical flow and deep learning are as follows: Step 1: The video acquisition module acquires video data from the disconnect switch, and the processor converts the video data into a continuous image sequence. Step 2: The first five frames of the image sequence are entered into the deep learning state recognition module for initial state recognition processing of the switch. Step 3: The deep learning state recognition module determines the initial state of the disconnector. If there is an abnormality, an alarm message is issued. If the state is normal, the image sequence proceeds to the next step. The algorithm flow corresponding to the deep learning state recognition module is described as follows: First, data preprocessing: load training images and labels, scale and crop the images, and map and transform the labels accordingly; Next, define the model: define the input layer, convolutional layer, pooling layer, and output layer; Then the loss function is calculated: classification loss, confidence loss, coordinate loss, and total loss are calculated; Next, train the model: define the optimizer and learning rate decay strategy, train on the training set, validate on the validation set, and save the model and weights; Final prediction: Load the model and weights, preprocess the input image, run the model to get the output results, perform post-processing including NMS and parsing the output results, and display or save the results; Step 4: The image sequence of the switch is entered into the optical flow recognition module for processing. It starts tracking based on the pre-set key points. If the trajectory of the key point movement is linear, it is considered to be a successful tracking and the switch operation status information is output. If the trajectory of the key point movement is non-linear, it is considered a tracking failure, and the last five frames of the image sequence are passed to the deep learning state recognition module, which outputs the switch operating status information.

2. The method for identifying the state of a disconnector based on optical flow and deep learning according to claim 1, characterized in that: The switch status recognition method based on optical flow and deep learning includes a deep learning status recognition module. This module only performs switch status detection on key frames of the image sequence, resulting in low computational complexity.

3. The method for identifying the state of a disconnector based on optical flow and deep learning according to claim 1, characterized in that: The switch status recognition method based on optical flow and deep learning includes an optical flow recognition module, which is sensitive to angle changes and has high detection accuracy.

Images