Hanging foreign matter cutting point positioning and tracking prediction method for overhead cable

Abstract

The invention discloses a mounting foreign matter cutting point positioning and tracking prediction method for an overhead cable, and the method comprises the steps: carrying out a series of image processing through employing an opencv library, including image preprocessing, edge detection and straight line fitting, and determining the coordinate positioning of a laser cutting point in an image; dynamic analysis and modeling are carried out on the motion characteristics of the overhead cable, a sliding window is fused into extended Kalman filtering by using the model, noise estimation and state variables are improved, the position of a laser cutting point is tracked and predicted, and visual delay compensation is carried out. Compared with an existing cable foreign matter aiming and tracking method, the cable foreign matter aiming and tracking method is higher in tracking accuracy and real-time performance.

Description

Technical Field

[0001] The present invention belongs to the field of removing and tracking foreign objects on transmission cables, and particularly relates to a method for positioning and tracking prediction of cutting points of foreign objects mounted on overhead cables. Background Art

[0002] A large number of transmission lines are erected between mountains, rivers, lakes and farmlands, with the characteristics of large span and relatively high altitude. With the improvement of productivity, the types and usage amounts of various floating objects are increasing. Different from harsh natural conditions or cable problems itself, the occurrence of faults can be reduced by optimizing the design of overhead cables. Foreign objects on cables cannot be avoided, and the problem can only be solved by timely removal.

[0003] At present, most front-line power workers still use traditional removal methods, which require working at heights, with low efficiency and dangerous live high-altitude operations. To seek a better removal method and improve the intelligence of power maintenance, using a laser emission device to carry out the removal work has become a new method to solve this fault problem. At present, the existing laser emission devices have poor automatic recognition and tracking effects in scenarios with background interference.

[0004] In order to increase the applicable range of the device and achieve the intelligence and automation of the device, a vision-based laser foreign object removal tracking and aiming system needs to be designed to realize the automatic positioning and tracking prediction of the cutting point of the laser emission device. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for positioning and tracking prediction of cutting points of foreign objects mounted on overhead cables, which can automatically identify and calculate the coordinates of the cutting point, and use predictive tracking to compensate for the time delay generated by image processing, and can greatly improve the real-time performance and accuracy of tracking.

[0006] The technical solution for realizing the purpose of the present invention is: a method for positioning and tracking prediction of cutting points of foreign objects mounted on overhead cables, including the following steps:

[0007] Step 1: Take frames of the real-time video and perform image preprocessing;

[0008] Step 2: Based on the linear morphological characteristics of the cable, detect the straight line after edge detection of the image, and use the RANSAC algorithm to fit the straight line to simplify the information of the cable in the image;

[0009] Step 3: Establish a two-dimensional coordinate system for the image to determine the position information of the cable in the image; through a custom image recognition algorithm, determine the coordinate position of the cutting point of the mounted foreign object in the image;

[0010] Step 4: Model the motion characteristics of the cable, and use the extended Kalman filter based on a sliding window to predict the cutting point to obtain the cutting point at the next moment.

[0011] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above method are implemented.

[0012] A computer-readable storage medium stores a computer program, and when the program is executed by a processor, the steps of the above method are implemented.

[0013] A computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0015] (1) Automatically locate the cutting point of the laser emission device. The recognition and positioning algorithm of the present invention is applicable to scenarios with foreground occlusion and non-complex interference backgrounds. After the laser foreign object removal device is set up, the picture includes the laser cutting point at the foreign object mounting location and the surrounding cables, and can automatically identify and calculate the coordinates of the cutting point, realizing the full automation of laser cutting;

[0016] (2) The present invention uses prediction and tracking to compensate for the time delay generated by image processing, and can greatly improve the real-time performance and accuracy of tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of the method for locating and tracking prediction of the cutting point of the foreign object mounted in the present invention.

[0018] Figure 2 It is an original diagram of an embodiment of the present invention.

[0019] Figure 3 It is a flowchart of the cutting point positioning algorithm of the present invention.

[0020] Figure 4 It is a flowchart of the prediction and tracking algorithm of the present invention.

[0021] Figure 5 It is a result diagram of the cutting point positioning algorithm in an embodiment of the present invention.

[0022] Figure 6 It is a comparison diagram of the predicted value and the true value in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The following further elaborates on the method provided in this application in combination with specific embodiments. Figure 1 It is a flowchart of the method for locating and tracking prediction of the cutting point of the foreign object mounted in the present invention, Figure 2 It is an original diagram of an embodiment of the present invention.

[0024] A method for positioning, tracking and predicting the cutting point of foreign objects hanging on overhead cables according to the present invention includes the following steps:

[0025] Step 1: Take frames from the real-time video and perform image preprocessing;

[0026] Step 2: Based on the linear morphological characteristics of the cable, detect the straight line after edge detection of the image, and use the RANSAC algorithm to fit the straight line to simplify the information of the cable in the image;

[0027] Step 3: Establish a two-dimensional coordinate system for the image to determine the position information of the cable in the image; through a custom image recognition algorithm, determine the coordinate position (u, v) of the cutting point of the hanging foreign object in the image;

[0028] Step 4: Model the motion characteristics of the cable, and use the extended Kalman filter based on a sliding window to predict the cutting point to obtain the cutting point at the next moment.

[0029] Among them, Steps 1 to 3 are the steps for positioning the cutting point of the foreign object hanging on the overhead cable, as Figure 3 shown.

[0030] Furthermore, the specific implementation method of Step 1 is: Take the current frame of the real-time video, and perform image enhancement, image grayscale conversion, and Gaussian noise reduction in sequence.

[0031] Furthermore, the specific implementation method of Step 2 is:

[0032] Step 2.1: Perform morphological denoising on the preprocessed image, and adopt the operation sequence of opening operation first and then closing operation;

[0033] Step 2.2: Process the image using the improved Canny edge detection. Use a 3×3 Sobel operator to calculate the gradient intensity and direction in four directions: horizontal, vertical, diagonal 45°, and 135°. The Sobel operators S x 、S y 、S a 、S b for the four directions are respectively:

[0034]

[0035] The gradient intensity of the pixel point takes the maximum value among the four directions. After edge thinning through non-maximum suppression, perform double-threshold processing, and use the Otsu algorithm to adaptively adjust the high and low thresholds; exclude weak edges through edge tracking hysteresis;

[0036] The improvement points are: ① Add the gradient intensity and direction in two directions of diagonal 45° and 135°; ② Use the Otsu algorithm to adaptively adjust the high and low thresholds.

[0037] Step 2.3: Use the RANSAC algorithm to fit a straight line. Classify the edge points in the edge detection result to obtain several edge point sets. Use the RANSAC algorithm to remove the "outliers" in the point set, that is, abnormal points or noise points, and perform straight line fitting.

[0038] The method for grouping the edge point sets described in Step 2.3 is as follows:

[0039] (1) Search for edge points and initialize the first group of point sets. Starting from the first column of the image, check row by row from top to bottom, and record the first edge point detected by Canny edge detection in this column, denoted as P 11 , and use it as the starting point of the first group of edge point sets;

[0040] (2) Traverse to the right column and complete the extraction of the first group of point sets. Continue to record edge points in the same way for the next column. The first Canny edge point in the k-th column is denoted as P 1k . After traversing all columns, obtain the first group of point sets, which participate in the fitting of the first straight line;

[0041] (3) Extract the second group of point sets. For the remaining points, perform the same column-by-column operation to obtain the second group of points participating in the fitting, P 21 , P 22 , P 2k ...;

[0042] (4) Output the grouping results of all edge points, and finally form several groups of edge point sets. Each group of point sets is composed of edge points selected from continuous pixel columns in the image.

[0043] Furthermore, the specific implementation method of the custom image recognition algorithm described in Step 3 is as follows:

[0044] Step 3.1: Take the upper left corner as the origin, with the x-axis and y-axis parallel to the two sides of the image plane respectively, and use (u, v) to represent the coordinate values of a certain pixel point.

[0045] Step 3.2: Arithmetically accumulate the gray values of the pixel points within a certain range at the upper and lower ends of the straight line. Let the gray matrix of the image be I, and I(u, v) corresponds to the gray value of the pixel point in the v-th row and u-th column. Take h1 pixel points above the straight line and h2 pixel points below as the vertical accumulation area, and record the gray integral value as S.

[0046]

[0047] Among them, h is the row number where the upper edge 1 of the cable is located, and S(u) represents the gray integral value of the u-th column within the upper and lower accumulation areas.

[0048] Step 3.3: Compare the integral difference between the current column and its adjacent column from left to right in sequence. If |S(u) - S(u - 1)| > T, where T is a threshold set to 2, it is determined that the gray integral of this column has a mutation, indicating the existence of a foreign object, and this point is recorded as the cutting point.

[0049] Through the positioning of the cutting point of the foreign object hanging on the overhead cable, the result is as Figure 5 shown.

[0050] Combined with Figure 4 , the tracking and prediction algorithm for the cutting point of the foreign object hanging on the overhead cable. According to the cable motion model, the state vector expression is

[0051]

[0052] where A is the amplitude, ω is the angular frequency, is the initial phase, B is the vertical offset, and dA is the amplitude change rate.

[0053] The state transition equation is

[0054]

[0055] where w k is the process noise, representing the random perturbation of the system.

[0056] The observation equation is where v k is the observation noise, conforming to a zero-mean Gaussian distribution.

[0057] The state transition matrix F

[0058]

[0059] The corresponding measurement matrix

[0060] Based on the analysis and modeling of the motion characteristics of the cable, on the basis of the known cable motion model, the combination algorithm of using a sliding window and an extended Kalman in the fourth step for predicting the cutting point at the next moment includes the following steps:

[0061] Step 4.1: Initialize the sliding window and fit the parameters of the cable motion model within the window;

[0062] Step 4.2: Obtain the coordinate values at the new moment k. If the sample size in the current sliding window has reached the window size, the window slides to fit new parameters. If k does not exceed the window size, return to Step 4.1;

[0063] Step 4.3: Adjust the size of the window based on the historical error; calculate the mean square error for the most recent W k frames. If the error e kLess than the lower threshold e low , reduce the window. Conversely, if the error e k is greater than the upper threshold e high , increase the window;

[0064] The window adjustment rule is as follows:

[0065]

[0066] where e k is the mean square error of the last W k frames, e low , e high are the error thresholds, W min , W max set the range of the window size, and ΔW is the unit step size of window adjustment.

[0067] Step 4.4: State prediction, using the optimal state estimate at the previous moment to predict the prior state estimate value

[0068]

[0069] where f(*) is the non-linear function of the state equation.

[0070] Step 4.5: State covariance prediction, using the optimal covariance matrix P at the previous moment k-1|k-1 to predict the prior covariance matrix P k|k-1 ,

[0071]

[0072] where F k-1 is the Jacobian matrix of the state transition function with respect to the state variables, and Q k is the covariance matrix of the process noise.

[0073] Step 4.6: Update the Kalman gain K k :

[0074]

[0075] where H k is the Jacobian matrix of the observation function with respect to the state variables, and R k is the covariance matrix of the observation noise.

[0076] Step 4.7: Update the state estimate value at time k

[0077]

[0078] where Zk is the observation value at time k, and h(*) is the non-linear function of the observation equation.

[0079] Step 4.8: Update the state estimation error covariance matrix P at time k k|k ,

[0080]

[0081] where, is the state posterior estimation.

[0082] Step 4.9: If the value at time k+1 is obtained, return to Step 4.2 to continue sliding the window and state prediction; otherwise, the prediction ends.

[0083] The prediction result of the embodiment video by the cutting point tracking and prediction algorithm is as Figure 6 , the diameter of the laser spot coverage area is at least 3 pixel points, so the prediction is accurate within one pixel point. After statistics, the accuracy rate is 99.2%. The aiming and tracking method for visual delay compensation of the present invention has both accuracy and real-time performance.

[0084] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for positioning and tracking prediction of cutting points of foreign objects attached to overhead cables, characterized in that It includes the following steps: Step 1: Take frames from the real-time video and perform image preprocessing; Step 2: Based on the linear morphological characteristics of the cable, detect the straight line after edge detection of the image, and use the RANSAC algorithm to fit the straight line to simplify the information of the cable in the image; Step 3: Establish a two-dimensional coordinate system for the image to determine the position information of the cable in the image; through a custom image recognition algorithm, determine the coordinate position of the hanging foreign object cutting point in the image; Step 4: Model the motion characteristics of the cable, and use the extended Kalman filter based on the sliding window to predict the cutting point to obtain the cutting point at the next moment.

2. The method for positioning, tracking and predicting the cutting point of foreign objects hanging on overhead cables according to claim 1, wherein The specific method of Step 1 is: Take the current frame of the real-time video and perform image enhancement, image grayscale conversion, and noise reduction in sequence.

3. The method for positioning, tracking and predicting the cutting point of foreign objects mounted on overhead cables according to claim 1, wherein The specific method of Step 2 is: Step 2.1: Perform morphological denoising on the preprocessed image, adopting the operation sequence of opening operation first and then closing operation; Step 2.2: Process the image using the improved Canny edge detection: Use a 3×3 Sobel operator to calculate the gradient intensity and direction in four directions: horizontal, vertical, diagonal 45°, and 135°. The Sobel operators S x , S y , S a , S b are respectively: The gradient intensity of the pixel point takes the maximum value in four directions; after edge thinning through non-maximum suppression, perform double-threshold processing, and use the Otsu algorithm to adaptively adjust the high and low thresholds; Step 2.3: Use the RANSAC algorithm to fit the straight line; classify the edge points in the edge detection result to obtain several edge point sets; use the RANSAC algorithm to remove the abnormal points or noise points in the point set and perform straight line fitting.

4. The method for positioning, tracking and predicting the cutting point of foreign objects hung on overhead cables according to claim 3, characterized in that, The method for grouping the edge point sets is: (1) Search for edge points and initialize the first set of point sets. Starting from the first column of the image, check row by row from top to bottom, and record the first edge point detected by Canny edge detection in this column, denoted as P 11 , and use it as the starting point of the first set of edge point sets; (2) Traverse the right column and complete the extraction of the first set of point sets. Continue to record the edge points in the same way for the next column. The first Canny edge point in the k-th column is denoted as P 1k ; After traversing all columns, the first set of point sets is obtained and participates in the fitting of the first straight line; (3) Extract the second set of point sets. For the remaining points, perform the same column-by-column operation to obtain the second set of points P participating in the fitting 21 , P 22 , P 2k ...; (4) Output all the edge point grouping results, and finally form several groups of edge point sets, and each group of point sets is composed of edge points selected from the continuous pixel columns in the image.

5. The method for positioning, tracking and predicting the cutting point of foreign objects mounted on overhead cables according to claim 1, characterized in that, The specific method of Step 3 is: Step 3.1: Take the upper left corner as the origin, with the x-axis and y-axis parallel to the two sides of the image plane respectively, and use (u, v) to represent the pixel point coordinate values; Step 3.2: Perform arithmetic accumulation on the gray values of the pixel points within a certain range at the upper and lower ends of the straight line; assume that the gray matrix of the image is I, and I(u, v) corresponds to the gray value of the pixel point in the v-th row and u-th column; take h1 pixel points above the straight line and h2 pixel points below the straight line as the vertical accumulation area, and record the gray integral value as S; Among them, h is the row number where the upper edge 1 of the cable is located, and S(u) represents the gray integral value of the u-th column within the upper and lower accumulation areas; Step 3.3: Compare the integral difference between the current column and its adjacent column from left to right in sequence. If |S(u) - S(u - 1)| > T, where T is the threshold, it is determined that the gray integral of this column has a mutation, indicating the existence of a foreign object, and this point is recorded as the cutting point.

6. The method for positioning, tracking and predicting the cutting point of foreign objects hung on overhead cables according to claim 5, characterized in that The value of T is 2.

7. The method for positioning, tracking and predicting the cutting point of foreign objects hung on overhead cables according to claim 1, wherein Analyze and model the motion characteristics of the cable. Based on the known cable motion model, in Step 4, the combined algorithm of the sliding window and the extended Kalman is used to predict the cutting point at the next moment. The steps are as follows: Step 4.1: Initialize the sliding window and fit the parameters of the cable motion model within the window; Step 4.2: Obtain the coordinate values at the new moment k. If the sample size in the current sliding window has reached the window size, the window slides to fit new parameters. If k does not exceed the window size, return to Step 4.1; Step 4.3: Adjust the size of the window based on historical errors; calculate the mean square error for the most recent W k frames. If the error e k is less than the lower threshold e low , reduce the window. Conversely, if the error e k is greater than the upper threshold e high , increase the window; Step 4.4: State prediction, using the optimal state estimate at the previous moment Predict the prior state estimate value Among them, f(*) is the non-linear function of the state equation; Step 4.5: State covariance prediction, using the optimal covariance matrix P at the previous moment k-1|k-1 Predict the prior covariance matrix P k|k-1 : Among them, F k-1 is the Jacobian matrix of the state transition function with respect to the state variables, and Q k is the covariance matrix of the process noise; Step 4.6: Update the Kalman gain K k : Among them, H k is the Jacobian matrix of the observation function with respect to the state variables, and R k is the covariance matrix of the observation noise; Step 4.7: Update the state estimate at time k where Z k is the observation value at time k, and h(*) is the non-linear function of the observation equation; Step 4.8: Update the state estimation error covariance matrix P at time k k|k , Among them, is the state posterior estimation; Step 4.9: If the value at time k+1 is obtained, return to Step 4.2 to continue sliding the window and predicting the state; otherwise, the prediction ends.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of Claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method described in any one of Claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method described in any one of Claims 1-7.

Images