Video jitter removal method for unmanned aerial vehicle overhead transmission line inspection

Abstract

The invention belongs to the technical field of target detection, and discloses a video jitter removal method for unmanned aerial vehicle overhead transmission line inspection, the method uses a feature point method to track and carry out electronic image stabilization, and the method comprises the following steps: firstly, based on GFTT and other feature point detection algorithms, extracting and analyzing feature points of continuous video frames in an original video; secondly, performing feature point tracking by using an optical flow method to obtain estimated values of feature points, then describing relative motion among the feature points by adopting affine transformation, and further performing linear processing to obtain an original motion track of the camera; performing motion smoothing processing on the original motion trail of the camera to obtain a stable motion trail, namely a smooth motion trail, representing the motion of the camera; and finally, calculating a compensation transformation matrix by comparing the original motion trail and the smooth motion trail of the camera, and compensating the frame of the original video so as to generate a video with stable motion.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a video stabilization method for unmanned aerial vehicle (UAV) inspection of overhead power transmission lines. Background Technology

[0002] For a long time, transmission line inspections have primarily relied on manual patrols. Patrol personnel trek through mountains and wilderness, walking along long corridors of power lines, requiring immense physical exertion. Moreover, the inspection efficiency is low, failing to meet the demands of the ever-increasing power grid mileage and increasingly frequent inspection cycles. Not only are the labor and time costs for patrol personnel high, but the physical labor intensity is also significant, and the safety risks are substantial. Furthermore, manual patrols have limited visibility and angles, resulting in low accuracy in observing and identifying problems, and are susceptible to subjective human factors. Some well-hidden faults may go undetected, leading to a high probability of missed inspections. While using manned helicopters for inspections has improved efficiency, the cost is too high to be routinely implemented. Therefore, it is evident that manual transmission line inspections are increasingly unable to meet the urgent needs of modern power grids as they transition towards large-area, high-voltage, intelligent, and lean operation and maintenance management.

[0003] The rapid development of drones as an aviation tool has brought a revolutionary opportunity to power line inspection. Drones have many advantages, such as small size and flexible operation, and are simpler, faster to deploy, and much cheaper than manned aircraft compared to other methods. Equipping drones with sensors such as high-resolution optical cameras allows them to reach areas inaccessible by traditional power line inspection methods. They can also conduct comprehensive and accurate aerial surveys of the power lines and related equipment from different directions and perspectives, providing clear and intuitive information for line maintenance and repair.

[0004] However, in actual power line inspection drone operations, new technical challenges arise due to the drone's hardware technology and the complex dynamic flight environment. Drones are inevitably affected by surrounding airflow disturbances, their own mechanical vibrations, and the constraints of gimbal stabilization accuracy, resulting in varying degrees of video jitter in the captured inspection videos. This video jitter not only degrades the viewing experience but also severely erodes the value of the data as crucial inspection information. The jitter blurs and obscures the video, leading to rapid, irregular displacements and geometric distortions of monitored targets within the field of view. This makes observing and interpreting critical details extremely difficult. For example, when accurately interpreting details such as minor damage to the power line surface, dirt or breakage on insulator strings, or the structural condition of power poles or towers, the information about the measured object is severely interfered with by video jitter, potentially leading to misjudgments and omissions. This impact on data accuracy caused by video jitter also directly affects the success of subsequent applications of intelligent analysis technologies based on computer vision. Summary of the Invention

[0005] The purpose of this invention is to propose a video stabilization method for unmanned aerial vehicle (UAV) inspection of overhead power transmission lines. This method uses feature point tracking for electronic image stabilization, which can better adapt to complex scenarios while maintaining high efficiency.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A video stabilization method for unmanned aerial vehicle (UAV) inspection of overhead power transmission lines includes the following steps: Step 1. Obtain the raw video data stream from the drone power line inspection. Step 2. Based on feature point detection and optical flow, perform motion estimation on the original video sequence to obtain the original motion trajectory of the camera; Step 3. Perform motion smoothing on the camera's original motion trajectory to obtain a smooth motion trajectory; Step 4. Calculate the compensation transformation matrix based on the difference between the camera's original motion trajectory and the smooth motion trajectory, and compensate the frames of the original video to generate a motion-stabilized video.

[0007] In addition, based on the above-mentioned video shading method for UAV overhead power line inspection, the present invention also proposes a computer device, which includes a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it implements the steps of the video stabilization method for UAV overhead power line inspection described above.

[0008] Furthermore, based on the aforementioned video stabilization method for UAV-based overhead power line inspection, this invention also proposes a computer-readable storage medium storing a program thereon; when executed by a processor, this program is used to implement the steps of the aforementioned video stabilization method for UAV-based overhead power line inspection.

[0009] The present invention has the following advantages: As described above, this invention discloses a video stabilization method for UAV-based overhead power line inspection. This method employs feature point tracking for electronic image stabilization. First, based on feature point detection algorithms such as GFTT, feature points in consecutive video frames of the original video are extracted and analyzed. Then, optical flow is used for feature point tracking to obtain estimated values ​​of the feature points. Next, affine transformation is used to describe the relative motion between feature points, followed by linearization to obtain the original motion trajectory of the camera. The original motion trajectory of the camera is then smoothed to obtain a stable motion trajectory representing the camera's motion, i.e., a smoothed motion trajectory. Finally, by comparing the original motion trajectory and the smoothed motion trajectory, a compensation transformation matrix is ​​calculated, and the frames of the original video are compensated, thereby generating a motion-stabilized video. The stabilized video, after stabilization using the proposed video stabilization method, can utilize target detection models such as YOLOv8 for automatic detection and identification of key power equipment targets. For example, it focuses on three key power equipment types: power lines, insulators, and utility poles. The target detection model detects and identifies targets in each frame of the stabilized video, enabling real-time target selection and classification. Attached Figure Description

[0010] Figure 1 This is a flowchart of a video decluttering method for unmanned aerial vehicle (UAV) inspection of overhead power transmission lines, as described in an embodiment of the present invention.

[0011] Figure 2 This is a schematic diagram of motion estimation in an embodiment of the present invention.

[0012] Figure 3 The precision-recall curve is obtained by performing target detection on a video that has been stabilized using the method of this invention.

[0013] Figure 4 This is the normalized confusion matrix obtained by performing target detection on a video after it has been stabilized using the method of this invention.

[0014] Figure 5 The training and validation loss and evaluation index curves are obtained by performing target detection on a video after it has been stabilized using the method of this invention.

[0015] Figure 6This is a visual annotation diagram of the target detection obtained from a video that has undergone stabilization processing using the method of this invention. Detailed Implementation

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Example 1

[0017] During the recording of power line inspection videos by drones, severe shaking can occur due to factors such as airflow disturbances and mechanical vibrations. This makes it difficult to identify critical power components, such as wires, insulators, and utility poles, directly affecting the validity of the inspection data and the accuracy of subsequent intelligent analysis. Traditional electronic image stabilization algorithms have poor adaptability to severe shaking and lack a quantitative evaluation system specifically for power line inspection scenarios.

[0018] To address the technical challenges of jitter removal in UAV power line inspection videos, this invention proposes a video de-jittering method based on feature point tracking and motion trajectory optimization. By integrating GFTT / ORB feature point detection, Lucas-Kanade optical flow tracking, affine transformation models, and moving average filtering techniques, it achieves accurate separation and compensation of jitter components. The method first employs a multi-algorithm collaborative feature point detection strategy, automatically switching to the ORB algorithm in low-texture scenes to ensure tracking stability. Second, it estimates inter-frame motion trajectories using optical flow and fits global motion using an affine transformation model. Finally, it utilizes moving average filtering to suppress high-frequency jitter and generates stable video through motion compensation. Furthermore, the method incorporates jitter quantification metrics, such as average jitter amplitude, to objectively evaluate the processing effect.

[0019] Compared to traditional image stabilization algorithms, this invention achieves adaptive optimization through hierarchical parameter configuration of core parameters, algorithm parameters, and adaptive parameters. This allows for dynamic adjustment of key parameters such as the smoothing radius and the number of feature points for different scenarios, including high-altitude line inspection and tower close-up inspection. Furthermore, this invention introduces an edge clipping mechanism to eliminate black borders generated after compensation while preserving the effective inspection area. While ensuring real-time performance, this method achieves an 80% elimination rate for minor jitter, significantly improving the accuracy of subsequent power component target detection.

[0020] like Figure 1 As shown, the video decluttering method for UAV inspection of overhead power transmission lines includes the following steps: Step 1. Obtain the raw video data stream of the drone power line inspection.

[0021] In this embodiment, the video was also preprocessed, including scaling it to 1080p and converting it to grayscale.

[0022] Step 2. Based on feature point detection and optical flow, perform motion estimation on the original video sequence to obtain the original motion trajectory of the camera.

[0023] like Figure 2 In step 2 of this embodiment, motion estimation of the video sequence for the original video includes: Use the GFTT algorithm or ORB algorithm to extract feature points from the original video frames.

[0024] Feature point tracking was performed using the Lucas-Kanade optical flow method to obtain the motion vectors of the feature points.

[0025] Based on the matched feature points, an affine transformation model is used to estimate the global motion between frames and obtain the motion parameters.

[0026] The following section provides a detailed explanation of the process of motion estimation for video sequences from the original video.

[0027] The process of motion estimation for the video sequence from the original video is as follows: Acquiring the original motion pattern of the camera is the first step in video stabilization. This invention uses a feature point tracking-based method to acquire the original motion pattern of the camera and incorporates computer vision methods.

[0028] First, it is necessary to find easily identifiable stable anchor points, i.e. feature points, in each video frame. To this end, this invention uses the GFTT corner detector to extract feature points in the original video frames. The GFTT corner detector uses the gradient covariance matrix of the pixel neighborhood and its eigenvalues ​​to find corner points in the image with drastic gray-scale changes. Since corner points have certain uniqueness, they are very suitable for tracking.

[0029] In addition, to balance speed and performance, when the number of valid feature points detected by the GFTT corner detector is less than 10, the ORB feature detector will be automatically invoked to extract feature points from the original video frame.

[0030] After identifying feature points in the current frame, the key is to locate the corresponding positions of these feature points in the next frame and use them to quantify inter-frame motion. The method of this invention is based on optical flow under the assumptions of constant brightness and local motion consistency to track feature points and calculate the displacement vector, i.e., the motion vector, of the feature points.

[0031] Optical flow methods can be used to obtain several pairs of feature points that can be matched between adjacent frames, as well as the displacements of these feature points. To unify these independent local motion information, a mathematical model is used to describe the global motion of the entire image frame. This invention employs an affine transformation model, which can describe complex combinations of translation, rotation, scaling, and shearing motions using six parameters.

[0032] Feature points matching between adjacent frames are obtained using optical flow. An affine transformation model is used to model inter-frame motion. From multiple pairs of matching feature points, the least squares method (often combined with RANSAC to remove outlier matches) is used to solve for the global affine transformation matrix T describing the relationship between the current frame and the previous frame, in order to estimate the global inter-frame motion and obtain motion parameters. The matrix T then accurately describes the uncorrected motion information before the actual camera motion at that moment.

[0033] Step 3. Perform motion smoothing on the camera's original motion trajectory to filter out high-frequency jitter components and obtain a smooth motion trajectory.

[0034] In step 3 of this embodiment, a moving average filtering algorithm is used to smooth the original motion trajectory of the camera, and the window size can be adaptively adjusted according to the degree of jitter.

[0035] Step 3 specifically involves: Step 2 yields a series of affine transformation matrices representing frame-by-frame motion through motion estimation. Step 3 decomposes these affine transformation matrices obtained from the motion estimation of the original video sequence in Step 2 to obtain the inter-frame translation. , and rotation angle .

[0036] By parameters , and Over time, the camera's original motion trajectory throughout the entire shooting process is constructed, i.e., the accumulated motion trajectory. .

[0037] Specifically, the The total displacement and rotation of the camera relative to the starting reference point at frame rate, i.e., the cumulative motion trajectory. for: .

[0038] Among them, components , , Calculated separately as follows:

[0039] in, Indicates the current frame number. This represents the index from the first frame to the current frame.

[0040] Original trajectory It accurately recorded all the camera's movements, but it included both the operator's smooth camera movements (intended movements) and unwanted high-frequency jitter caused by factors such as airflow and vibration. To extract the original motion trajectory... The method of this invention uses moving average filtering technology to separate the true intentional motion of the camera and filter out unnecessary jitter.

[0041] Using a low-pass filter to analyze the components of the original motion trajectory , and Perform moving average filtering to obtain the smoothed trajectory components. , and Thus, a smooth motion trajectory is obtained. .

[0042] The low-pass filter calculates the trajectory points within a fixed width. The arithmetic mean within the sliding window is used to smooth the components of the original motion trajectory, thereby generating a smoother, more ideal motion trajectory. The smoothing radius represents the smoothed horizontal translation component. for:

[0043] The same method applies to... and Obtain the smoothed trajectory components. Among them, the smoothing radius... The choice is a crucial balancing point. If the value is too small, the vibration filtering will be insufficient. If it is too large, it may cause the intended movement to be overly smooth, creating a sense of delay.

[0044] Step 4. Calculate the compensation transformation based on the difference between the original motion trajectory and the smooth motion trajectory. That is, calculate the compensation transformation matrix based on the difference between the camera's original motion trajectory and the smooth motion trajectory, and compensate the frames of the original video to generate a motion-stabilized video.

[0045] A smooth motion trajectory that represents the camera's true motion intention was obtained. And recorded the original motion trajectories of all movements. Finally, step 4, image compensation, is the final step in video stabilization. The purpose of this stage is to perform precise geometric correction on each frame of the original video based on the difference between the original motion trajectory and the smoothed motion trajectory, making it appear as if it follows a smooth trajectory. The photos were taken in the same way.

[0046] In this embodiment, the geometric correction performed on the original video in step 4 specifically includes compensation transformation and edge cropping.

[0047] Frame compensation of the original video includes: Based on the compensation transformation matrix, the original video frame is distorted using an affine transformation to obtain the compensated video frame.

[0048] Then, edge cropping is performed on the compensated video frames to remove invalid areas.

[0049] The following section details the process of frame compensation for the original video.

[0050] The process of compensating for frames in the original video is as follows: Calculate the amount of jitter that needs to be eliminated in each frame, i.e., the camera's original motion trajectory. With smooth motion trajectory The difference between them yields the motion parameters used to compensate for the current frame. , , They are respectively:

[0051] Based on compensated motion parameters , , Calculate the compensation transformation matrix The compensation transformation matrix describes what geometric transformation should be applied to the k-th frame of the original video to bring it back to the ideal smooth motion path.

[0052] By calling the `cv2.warpAffine` function from the OpenCV image processing library, a compensation transform is applied to the original frames to compensate for the original video frames and generate motion-stabilized video. This operation generates stabilized image frames by repositioning and interpolating pixels. : .

[0053] in, This represents the k-th frame of the original video. The dimensions of the output image. It specifies how to handle areas outside the image boundaries that may arise due to the transformation. This represents the stabilized image frame.

[0054] After stabilization, the video may contain invalid regions (often represented by black borders) due to the geometric transformations performed on it. To achieve a good visual effect, the stabilized frames are typically trimmed to remove these invalid regions and retain as much useful content as possible.

[0055] In step 4 of this embodiment, the video jitter is further quantitatively analyzed by calculating the average jitter amplitude and jitter level.

[0056] In this embodiment, the average jitter amplitude is calculated for the original video and the stable video, thereby objectively evaluating the jitter of the original input video and quantifying the actual effect of the method of the present invention.

[0057] The average jitter amplitude is calculated based on the average Euclidean distance between the motion vectors of different frames. The jitter level is classified into slight, moderate, noticeable, severe, and extreme jitter based on the average jitter amplitude.

[0058] The core quantitative indicator is the average jitter amplitude. It measures the average severity of relative translation between video frames.

[0059] When calculating the average jitter amplitude, first determine each frame starting from the second frame. Compared to the previous frame The original translation vector The Euclidean distance, i.e., the instantaneous jitter amplitude. :

[0060] Then all instantaneous jitter amplitudes Take the arithmetic mean to obtain the average jitter of the entire video clip. :

[0061] in, This represents the total number of frames in the video. The larger the value, the more severe the video jitter.

[0062] In addition to assessing the amplitude of jitter, this embodiment can also use heuristic methods to determine whether the jitter is high-frequency vibration or low-frequency shaking from the perspective of frequency. Combining these quantitative data, the overall jitter of the video is divided into several relatively intuitive levels for easier-to-understand jitter stability judgment.

[0063] These quantization results not only demonstrate the image stabilization effect, but also provide important basis for optimizing algorithm parameters.

[0064] Artificial intelligence technology, with deep learning at its core, has achieved a major breakthrough in the field of computer vision. Object detection models are becoming increasingly mature and their performance is continuously improving, providing unprecedented technical support for the automatic analysis of power line inspection video content and the automatic identification of critical power equipment components.

[0065] Among numerous advanced target detection models, the YOLO series stands out for its unique single-stage detection framework design, achieving a balance between detection speed and accuracy. It is considered a classic and highly efficient model in the field of real-time target detection. Applying high-performance and efficient target detection models, such as YOLOv8, to the analysis of power line inspection videos enables rapid and accurate location and identification of critical power facility components. This significantly reduces the workload of manually interpreting large volumes of inspection videos, improving the efficiency and effectiveness of inspection work. Furthermore, it maximizes the objectivity and uniformity of the videos, enhancing defect identification accuracy and the level of intelligence in inspection processes.

[0066] Based on this, and addressing the jitter problem widely used in UAV power line inspection videos that severely compromises data quality and subsequent processing and analysis, this invention delves into and applies video stabilization methods to improve video data quality. This provides stable and clear video data as a visual foundation, which can then be combined with the YOLOv8 target detection model to achieve automatic, intelligent, and accurate identification of power components in UAV power line inspection scenarios. This effectively solves the jitter problem in UAV power line inspection videos, avoids many drawbacks of traditional manual inspection methods, and significantly improves the efficiency and intelligence level of power grid operation and maintenance management. It has significant strategic importance and application value for ensuring the safe, stable, and reliable operation of the power energy supply system.

[0067] In addition, to verify the effectiveness of the method proposed in this invention, the following specific experiments are also provided: In this embodiment, target detection is performed on a video stabilized using the video stabilization method proposed in this invention, and the result is compared with several mainstream target detection algorithms to comprehensively evaluate the performance of the method in different metrics. Analysis of the results from the comparative and ablation experiments will further verify the effectiveness of the method and its improvement in target detection accuracy.

[0068] To ensure the fairness of the experiment, all uses in this invention were performed on a Linux system equipped with a 13th Gen Intel® Core™ i9-13900K CPU and dual NVIDIA GeForce RTX 4090 GPUs.

[0069] The parameter configurations are shown in Table 1. Among them, the core parameters directly affect the algorithm performance, the algorithm parameters affect the internal configuration of a specific algorithm, and the adaptive parameters are dynamically adjusted according to the input video features.

[0070] Table 1 Parameter Configuration

[0071] The function of each parameter in Table 1 is explained below.

[0072] The smoothing_radius parameter controls the window size of the moving average filter, determining the algorithm's ability to suppress high-frequency jitter. A larger smoothing_radius value, such as 30, can effectively remove drone jitter but will cause delays in dynamic panning; a smaller smoothing_radius value can better preserve dynamic details but cannot effectively suppress high-frequency jitter.

[0073] `border_crop` is used to crop the black border area at the edge of the corrected video to avoid image distortion. For a 1080p video, cropping by 50 pixels retains approximately 91.4% of the effective area. Excessive cropping may remove the target, affecting subsequent detection.

[0074] `maxCorners` is used to constrain the maximum number of feature points detected by the GFTT algorithm, balancing computational efficiency and tracking accuracy. In power line inspection scenarios, 200 points can cover 95% of the critical area; when the number of points exceeds 300, the processing time per frame increases from 120ms to 175ms.

[0075] `winSize` is the size of the search window for the optical flow algorithm, and its selection affects the tracking speed and stability of feature points. A larger search window, such as a 21×21 window, has stronger noise resistance but poorer responsiveness to tracking fast-moving objects. A smaller search window can capture pixel features with small displacements but is more sensitive to noise.

[0076] `min_valid_points` is a threshold value representing the critical number of valid feature points. When the number of valid feature points falls below this threshold, the detector is switched to the ORB feature detector. This avoids the failure of the GFTT corner detector in low-texture scenes such as skies and smooth walls. The ORB descriptor improves the robustness of GFTT by introducing BRIEF. The ORB algorithm takes 85ms / frame, which is 30% faster than the GFTT algorithm, but it requires subsequent RANSAC filtering to reduce false matches.

[0077] Based on the YOLOv8n pre-trained model, the training parameters are configured through the model.train() interface in this embodiment, and the specific parameters are shown in Table 2.

[0078] Table 2 Training Parameters

[0079] The hardware and software environment used for training is shown in Table 3.

[0080] Table 3 Hardware and software environment

[0081] The key evaluation metrics are: the precision-recall curve, which reflects the precision of the recall under various recall conditions; the area under the curve (AP) can be used as an evaluation metric for model performance; and mAP, or average AP, can be used to comprehensively evaluate the model's detection performance for multiple categories.

[0082] Figure 3 This is the precision-recall curve for this invention. The vertical axis represents precision, and the horizontal axis represents recall. In the legend, "wire" represents a wire, "pole" represents a tower, and "insulator" represents an insulator. "All classes 0.943 mAP@0.5" represents the average precision for all classes. @0.5 indicates that the cross-union ratio (CUP) threshold used in calculating mAP is 0.5. In object detection, a predicted bounding box must have an CUP greater than or equal to 0.5 with the ground truth bounding box to be considered a valid positive example.

[0083] The curves show the detection performance of different types of models and the differences in detection performance between different models. Figure 3 The AP (Average Per Scale) for utility poles is 0.995, for insulators it is 0.963, and for wires it is 0.870. The overall mAP@0.5 is 0.943, indicating that the model performs well in detecting utility poles and insulators, but poorly in detecting wires.

[0084] The confusion matrix is ​​fundamental to the model's classification accuracy. It visually demonstrates the model's ability to identify wires, insulators, and utility poles by statistically analyzing the number of confused predictions for each category during the classification process. The diagonal elements represent the number of correctly classified categories, while the off-diagonal elements represent the number of confused categories. After normalization, the number of elements in the matrix represents a proportion, not an absolute number, used to compare the confusion rates of different categories.

[0085] Figure 4 A normalized confusion matrix is ​​used to identify power transmission facilities. "wire" represents a conductor, "pole" represents a tower, "insulator" represents an insulator, and "background" represents the video background. The vertical axis of this matrix represents the predicted object, and the horizontal axis represents the true value. This matrix indicates the probability that a true object is identified as another object under the model designed in this invention. Figure 4 As shown, the accuracy rate for identifying power lines was 0.82, while the accuracy rates for identifying power poles and insulators were 0.95 and 0.96, respectively. This may be related to the fact that the power lines in the images were too long and thin to be identified.

[0086] Training and Validation Loss and Metrics Curves monitor the model training process. The training loss curve shows the change of the loss function during training. The validation loss curve is used to judge the model's performance on the validation set and determine whether overfitting exists. The evaluation metric curves show the changes of metrics such as precision, recall, and average precision during training to determine whether the model's performance has improved.

[0087] Figure 5 This section describes the changes of multiple training or evaluation metrics of the model with the number of iterations. Specifically, `train / box_loss` represents the bounding box loss during the training process; `train / cls_loss` represents the object classification loss during training; `train / dfl_loss` represents the distribution focus loss during training; `metrics / precision(B)` represents the training set precision, measuring the model's accuracy during training; `metrics / recall(B)` represents the training set recall, measuring the model's recall during training; `val / box loss` represents the validation set object bounding box regression loss; `val / cls loss` represents the validation set object classification loss; `val / dfl loss` represents the validation set distribution focus loss; `metrics / mAP50(B)` represents the validation set mAP@0.5 precision curve; and `metrics / mAP50-95(B)` represents the average mAP convergence effect of the validation set mAP@0.5-0.95 under different intersection-union (IU) thresholds.

[0088] like Figure 5 As shown, the training loss curves are generally decreasing, indicating that the loss is constantly being optimized during the model's learning and training process. The validation loss curve tends to stabilize in the later stages of training, and the difference between the validation loss curve and the training loss curve is not significant, indicating that the model is not severely overfitting. The precision, recall, and mAP evaluation metrics curves all show varying degrees of upward trend during training and tend to stabilize, indicating that the model performance is continuously improving and reaches a relatively stable high level in the later stages.

[0089] Post-processing and visualization of detection results: The raw results of object detection may contain redundant boxes and low-confidence predictions, requiring post-processing to improve accuracy. The YOLOv8 model outputs the object category and confidence score for each detection box. Post-processing involves setting a threshold to filter out low-confidence detection boxes and retaining high-confidence results. In this embodiment, considering the difficulty in distinguishing thin wires, conf_threshold is set to 0.25 to balance recall and precision.

[0090] To suppress redundant detection boxes for the same target, Generalized Intersection over Union (GIoU) ​​is used as the metric for Non-Maximum Suppression (NMS). In this embodiment, iou_threshold is set to 0.45. When the IoU value of two detection boxes is greater than 0.45, the detection box with lower confidence will be suppressed. The calculation formula is expressed as:

[0091] Here, A and B represent two bounding boxes, |A| and |B| represent their areas, and |A∩B| represents their intersection area.

[0092] Target detection in videos may suffer from jitter or discontinuity. To address this, BotSORT is introduced to correlate detection results across consecutive frames. This algorithm combines appearance features and motion information, making it suitable for long-term stable tracking of power equipment. The processing flow is as follows: first, feature extraction is performed using a YOLOv8 backbone network; then, Kalman filtering is used to predict the target's position in consecutive frames; finally, data correlation is performed based on appearance features and motion information.

[0093] To facilitate user understanding and analysis, the detection results need to be visualized based on video frames, and relevant data need to be statistically analyzed.

[0094] For each detected target in the final output, the system performs visibility annotation on the corresponding video frame, such as... Figure 6 As shown, it includes the bounding box of the target's location, the category label of the target's classification name, and the confidence score of the detection result, and can help users quickly locate key electrical components within a video frame. Figure 6 This is a diagram illustrating a real-world application scenario. Here, `id` represents the target information for wire tracking, `wire` represents the wire, and the following number indicates the wire's recognition probability. `pole` represents the tower, and the following value indicates the tower's recognition probability.

[0095] In addition, there is a results display module that counts the number of detected targets appearing in the video or a segment of a video, as well as the number of each target. This value can be used as the distribution characteristics of the circuit components or the defect density characteristics of the fault type.

[0096] Example 2 This embodiment 2 describes a computer device that includes a memory and one or more processors.

[0097] The memory stores executable code, which, when executed by the processor, is used to implement the steps of the video stabilization method for UAV overhead power line inspection in Embodiment 1 above.

[0098] In this embodiment, the computer device can be any device or apparatus with data processing capabilities, and will not be described in detail here.

[0099] Example 3 This embodiment 3 describes a computer-readable storage medium storing a program that, when executed by a processor, implements the steps of a video stabilization method for unmanned aerial vehicle (UAV) inspection of overhead power transmission lines.

[0100] The computer-readable storage medium can be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc.

[0101] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

[0102] Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention and not intended to limit it. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the specific embodiments of the present invention, but such modifications or alterations are all within the scope of protection of the pending claims.

Claims

1. A method for video deblurring for unmanned aerial vehicle (UAV) overhead transmission line inspection, characterized in that, The method comprises the following steps: Step 1. Obtain the unmanned aerial vehicle power inspection video data stream, i.e. the original video; Step 2. Perform motion estimation on the original video based on feature point detection and an optical flow method to obtain the original motion trajectory of the camera; Step 3. Perform motion smoothing processing on the original motion trajectory of the camera to obtain a smoothed motion trajectory; Step 4. Calculate a compensation transformation matrix based on the difference between the original motion trajectory and the smoothed motion trajectory of the camera, and compensate the frames of the original video to generate a motion-stable video.

2. The video deblurring method for unmanned aerial vehicle overhead transmission line inspection according to claim 1, characterized in that: in step 2, the motion estimation on the original video comprises: extracting feature points in the original video frames using a GFTT algorithm or an ORB algorithm; tracking the feature points using a Lucas-Kanade optical flow method to obtain the motion vectors of the feature points; based on the matched feature points, estimating the inter-frame global motion using an affine transformation model and obtaining the motion parameters.

3. The video deblurring method for unmanned aerial vehicle overhead transmission line inspection according to claim 1, characterized in that: in step 2, the process of the motion estimation on the original video comprises: extracting feature points in the original video frames using a GFTT corner detector, and when the number of detected valid feature points is less than 10, extracting feature points in the original video frames using an ORB feature detector to balance the speed and performance; tracking the feature points based on the Lucas-Kanade optical flow method under the assumption of constant brightness and local motion consistency to calculate the displacement vectors, i.e. the motion vectors, of the feature points; based on the matched feature points between adjacent frames obtained by the optical flow method, using an affine transformation model to solve the affine transformation matrix between the current frame and the previous frame using a least squares method to estimate the inter-frame global motion and obtain the motion parameters.

4. The video deblurring method for unmanned aerial vehicle overhead transmission line inspection according to claim 1, characterized in that: in step 3, the original motion trajectory of the camera is smoothed by a sliding average filtering algorithm.

5. The video deblurring method for unmanned aerial vehicle overhead transmission line inspection according to claim 3, characterized in that: in step 3, the smoothed motion trajectory is obtained by: Decompose the affine transformation matrix obtained from the motion estimation of the video sequence of the original video in step 2 to obtain the inter-frame translation amount , and the rotation angle ; By accumulating the parameters , , over time, the original motion trajectory of the camera during the entire shooting process, i.e. the cumulative motion trajectory , is constructed No. The total displacement and rotation of the camera relative to the starting reference point at frame rate, i.e., the cumulative motion trajectory. for: ； wherein the components are calculated as follows: ； ； ； wherein, represents the current frame number, represents the index from the first frame to the current frame; The components of the original motion trajectory are filtered by a low-pass filter , and The smoothed trajectory components , and are obtained by using a sliding average filter, and then the smoothed motion trajectory is obtained. The low-pass filter calculates the trajectory points within a fixed width. The arithmetic mean within the sliding window is used to smooth the individual components of the original motion trajectory, where The smoothing radius represents the smoothed horizontal translation component. for: 。 6. The video deblurring method for unmanned aerial vehicle overhead transmission line inspection according to claim 1, characterized in that: in step 4, the compensation of the frames of the original video comprises: using affine transformation to warp the original video frames according to the compensation transformation matrix to obtain the compensated video frames; then performing edge cropping on the compensated video frames to remove invalid areas.

7. The video deblurring method for unmanned aerial vehicle overhead transmission line inspection according to claim 5, characterized in that: in step 4, the process of the compensation of the frames of the original video comprises: The amount of jitter to be eliminated for each frame, i.e. the original motion trajectory of the camera, is calculated The difference between the smoothed motion trajectory and the original motion trajectory, resulting in motion parameters for compensating the current frame , , are respectively: ； ； ； According to the compensation motion parameters , , Calculate the compensation transformation matrix ; by calling the v2.warpAffine function of the OpenCV image processing library, the compensation transformation is applied to the original frames to compensate the frames of the original video and generate a motion-stable video. ； wherein, denotes the k-th image frame of the original video, is the size of the output image, is a mode for setting a processing of a region other than the image boundary generated by the transformation, denotes the stabilized image frame.

8. The video de-shaking method for unmanned aerial vehicle overhead transmission line inspection according to claim 1, characterized in that, in step 4, the video shaking is quantitatively analyzed by calculating the average shaking amplitude and the shaking level; wherein the average shaking amplitude is calculated based on the average Euclidean distance of inter-frame motion vectors.

9. A computer device comprising a memory and one or more processors, the memory having stored therein executable code, the computer device characterized in that, The processor executes the executable code to implement the steps of the video de-shaking method for unmanned aerial vehicle overhead transmission line inspection according to any one of claims 1 to 8.

10. A computer-readable storage medium having stored thereon a program, characterized in that, The program is executed by the processor to implement the steps of the video de-shaking method for unmanned aerial vehicle overhead transmission line inspection according to any one of claims 1 to 8.

Images