Automatic obstacle-clearing tracking aiming method and system for unmanned aerial vehicle-mounted laser

Abstract

The invention provides an unmanned aerial vehicle laser automatic obstacle clearance tracking aiming method and system, and belongs to the technical field of intelligent power grid operation and maintenance. The method comprises the following steps: acquiring a real-time video stream of a power transmission line channel by using visual acquisition equipment, and acquiring an obstacle target distance in cooperation with a laser ranging module; coarse aiming of the floating and hanging object is realized through image recognition and cooperative control; in the process of emitting high-energy laser to clear obstacles, a self-adaptive tracking step based on ablation feedback is executed, that is, ablation smog and spark states of a target area are monitored in real time, a visual confidence factor and a ranging fluctuation factor are calculated, and an ablation disturbance index is constructed; and dynamically adjusting the ratio of the visual feedback weight to the motion prediction weight in the tracking algorithm based on the index. When smoke interference is serious, the system is automatically switched to an inertia keeping mode, and locking is maintained by means of a historical movement track. The technical problems that in the laser obstacle clearing process, due to smoke shielding, visual tracking fails, and due to target thermally-induced flutter, the aiming precision is reduced are solved.

Description

Technical Field

[0001] This invention relates to the field of smart grid operation and maintenance technology, and in particular to an automatic obstacle clearing, tracking, and aiming method and system for UAV-borne lasers. Background Technology

[0002] The safety of overhead transmission lines is a crucial aspect of the stable operation of smart grids. Floating debris such as kites, plastic films, and dust nets are major hazards that can cause short circuits, trip power lines, and endanger grid safety. With the advancement of drone technology, obstacle-clearing drones equipped with high-energy lasers have emerged. These drones utilize high-energy laser beams to ablate floating debris over long distances. Leveraging their advantages of non-contact operation, rapid response, and large operating radius, they have gradually replaced traditional manual live-line work, becoming the mainstream equipment for power maintenance.

[0003] Existing UAV laser obstacle removal systems typically employ image-based visual servo tracking technology. However, during laser obstacle removal, the high-energy laser ablation of obstacles produces dense smoke and open flames, leading to the failure of visual feature extraction. Simultaneously, the recoil force generated by the heated and vaporized obstacles causes high-frequency, irregular flutter. The bandwidth of existing visual feedback systems is often insufficient to cover this high-frequency flutter and cannot maintain target lock even when visual features are lost, easily causing gimbal drift or accidental damage to background guide lines. Summary of the Invention

[0004] The main objective of this invention is to solve the technical problems existing in the current UAV laser obstacle clearing technology, such as the contradiction between visual feedback and smoke ablation, the mismatch between control bandwidth and thermal flutter frequency, and the easy damage to wires in dynamic scenes. The invention provides an automatic obstacle clearing tracking and aiming method and system for UAV-borne lasers that binds environmental perception and motion control.

[0005] The specific technical solution adopted in this invention is as follows:

[0006] In a first aspect, the present invention provides an automatic obstacle clearing tracking and aiming method for an unmanned aerial vehicle (UAV)-borne laser, which is applied to an obstacle clearing system comprising an UAV flight platform, an airborne gimbal, a visual acquisition device, and a laser emitting device. The method is characterized by comprising the following steps:

[0007] The visual acquisition device acquires real-time monitoring video streams of the power transmission line channel, and controls the laser emitting device to emit ranging lasers to obtain distance information of obstacle targets;

[0008] Image detection is performed on the real-time monitoring video stream to identify hanging obstacles on the transmission line, and the pixel center position of the obstacle in the image coordinate system is calculated.

[0009] Based on the pixel center position and the distance information, a cooperative control command is generated to drive the UAV flight platform and the airborne gimbal to move, so that the laser aiming optical axis is aligned with the obstacle target;

[0010] The method also includes an adaptive tracking step based on ablation feedback:

[0011] During the process of controlling the laser emitting device to emit high-energy lasers for obstacle clearing, the ablation state of the obstacle target area is monitored in real time, and the visual confidence factor characterizing the visual observation quality and the distance measurement fluctuation factor characterizing the distance measurement stability are calculated.

[0012] An ablation disturbance index is constructed based on the visual confidence factor and the ranging fluctuation factor.

[0013] Based on the ablation perturbation index, the trust weights of visual observations and motion model predictions in the tracking control algorithm are dynamically adjusted; as the ablation perturbation index increases, the trust weight of the visual observations is reduced.

[0014] When the ablation disturbance index exceeds a preset threshold, the airborne gimbal is controlled to enter the inertial hold mode, and the motion trend of the airborne gimbal is extrapolated using a motion model until the ablation disturbance index returns to the normal range.

[0015] Optionally, the step of dynamically adjusting the trust weights of visual observations and motion model predictions in the tracking control algorithm based on the ablation perturbation index specifically includes:

[0016] A Kalman filter is used as the tracking control algorithm.

[0017] Establish the mapping relationship between the ablation perturbation index and the Kalman filter measurement noise covariance matrix;

[0018] As the ablation perturbation index increases, the value of the measurement noise covariance matrix increases according to a preset growth function;

[0019] The Kalman gain is updated using the adjusted measurement noise covariance matrix, thereby reducing the dependence of system state updates on visual observations.

[0020] Optionally, the calculated visual confidence factor specifically includes:

[0021] Extract image texture features within the bounding box region of the obstacle target and calculate the rate of change of texture complexity between the current frame and the previous frame;

[0022] The number of bright pixels within the bounding box area is counted. If the growth rate of the number of bright pixels exceeds a preset threshold, it is determined that spark interference has occurred.

[0023] The visual confidence factor is calculated by combining the texture complexity change rate and the number of highlighted pixels using a normalization function.

[0024] Optionally, the calculation of the ranging fluctuation factor specifically includes:

[0025] Establish a sliding time window for laser ranging data and collect distance values ​​from N consecutive sampling points; N is a positive integer.

[0026] Calculate the standard deviation of the N sampling points and compare the standard deviation with a preset system noise benchmark value;

[0027] If the standard deviation is greater than the system noise reference value, the ratio of the excess amplitude is determined as the ranging fluctuation factor.

[0028] Optionally, the method further includes a feedforward compensation step based on thermally induced flutter:

[0029] The current output power of the laser emitting device is obtained, and the classification of the obstacle target is determined using image recognition.

[0030] According to the classification, the corresponding pre-calibrated thrust coupling coefficient and pre-calibrated surface density parameters are retrieved from the preset database;

[0031] The vaporization recoil force is estimated based on the output power and the pre-calibrated thrust coupling coefficient, and the theoretical mass is estimated based on the visual projection area and the pre-calibrated surface density parameter.

[0032] The physical suspension length from the hanging point to the geometric center of the obstacle target is calculated using image recognition, the natural oscillation frequency of the obstacle target is estimated based on the principle of the simple pendulum model, and the equivalent stiffness coefficient is calculated in reverse by combining the theoretical mass.

[0033] The moment when the laser emitting device turns on to emit light is set as time zero. The theoretical oscillation trajectory of the obstacle target under the action of the vaporization recoil force is calculated using a dynamic model that includes the vaporization recoil force, theoretical mass and equivalent stiffness coefficient.

[0034] A compensation component with the opposite phase to the theoretical oscillation trajectory is generated and superimposed on the gimbal attitude control command.

[0035] Optionally, the method further includes an intermittent pulse modulation step:

[0036] When the ablation disturbance index exceeds the preset safety fuse threshold, a laser pause command is generated to temporarily block the energy output of the laser emitting device.

[0037] Image data is continuously acquired during the period when laser output is stopped, and the recovery of the visual confidence factor is continuously monitored;

[0038] When the visual confidence factor rises back to the observable range and lasts for more than the preset anti-shake delay, a laser recovery command is generated, and the aiming point is recalibrated according to the current pixel center position.

[0039] Optionally, the method includes a decoupling control step between the UAV position and the gimbal attitude:

[0040] Calculate the deviation vector between the pixel center position and the image field of view center;

[0041] The deviation vector is decomposed into a low-frequency, large-amplitude offset component and a high-frequency, small-amplitude jitter component.

[0042] The large offset component is used to generate a UAV flight control signal, which drives the UAV flight platform to perform position translation to maintain coarse alignment.

[0043] The gimbal attitude control command is generated using the small-amplitude jitter component, which drives the airborne gimbal to perform fine rotation to eliminate residual deviations.

[0044] Optionally, the step of performing image detection on the real-time monitoring video stream further includes a power transmission line protection mechanism:

[0045] While identifying the obstacle target, linear features of the power transmission lines in the background are extracted;

[0046] Calculate the minimum pixel distance between the edge contour of the obstacle target and the power transmission line;

[0047] Predict the location of the laser spot at the next moment using a prediction algorithm;

[0048] If the distance between the landing point and the power transmission line is less than a preset safety protection threshold, the triggering permission of the laser emitting device is locked.

[0049] Optionally, the method further includes a multi-objective intelligent sorting step:

[0050] When the image detection results contain multiple hanging obstacles, the minimum distance between each obstacle and the key hardware of the transmission line is calculated as the risk level.

[0051] Prioritize locking onto obstacles with the highest risk level and the smallest visual projection area at the attachment point location;

[0052] After clearing the current obstacle, the memory module automatically retrieves the position information of the next highest priority obstacle and drives the airborne gimbal to turn quickly.

[0053] Secondly, the present invention provides an automatic obstacle clearing and tracking aiming system for an unmanned aerial vehicle (UAV) equipped with a laser, characterized in that it comprises:

[0054] The perception module, including a visual acquisition device and a laser ranging module, is used to collect environmental data;

[0055] The image processing unit is used to perform image detection on real-time monitoring video streams, identify obstacle targets, and calculate pixel center positions.

[0056] The ablation state analysis unit is used to monitor the ablation state of obstacles and targets in real time during laser strikes, calculate the visual confidence factor and ranging fluctuation factor, and construct the ablation disturbance index.

[0057] An adaptive controller is used to dynamically adjust the ratio of visual feedback weight to motion prediction weight in the tracking control algorithm based on the ablation disturbance index, and generate gimbal attitude control commands.

[0058] The actuators, including the drone flight platform and airborne gimbal, are used to respond to control commands to achieve tracking and aiming;

[0059] When the ablation disturbance index exceeds a threshold, the adaptive controller enters an inertial hold mode to resist smoke interference.

[0060] The present invention has achieved the following beneficial effects:

[0061] This invention links the severity of the working environment with the trust strategy of the control system, innovatively introducing the ablation disturbance index as a core evaluation indicator to dynamically adjust the weight allocation of visual feedback and motion prediction. This mechanism establishes an adaptive control logic chain: the system first calculates in real time the visual confidence factor, representing observation quality, and the ranging fluctuation factor, representing signal stability; when the laser strike generates dense smoke causing a sharp increase in the ablation disturbance index, the system identifies that the visual data is no longer reliable, and automatically reduces the weight of visual measurements in the Kalman filter, smoothly switching to inertial hold mode; in this mode, the gimbal maintains its trajectory based on the motion vector of the target before entering the smoke. This invention achieves a smooth switch from visual feedback tracking to model prediction tracking by quantifying environmental ablation disturbances. During the blind zone of visual failure, the state prediction extrapolation function of the Kalman filter is used to maintain the gimbal's trajectory, effectively preventing gimbal drift.

[0062] This invention addresses the physical phenomenon of thermal flutter as a dynamic response governed by physical laws, rather than unpredictable random noise. Therefore, to solve this physical challenge affecting obstacle clearing efficiency, this invention introduces a feedforward compensation mechanism based on a physical model. By identifying material properties and estimating the vaporization recoil force, the system predicts the target's oscillation trend before displacement occurs using a dynamic model and superimposes a reverse compensation component into the control commands. This allows the laser spot to closely follow and adhere to the violently fluttering target, overcoming the lag of traditional visual feedback, significantly shortening clearing time, and improving energy utilization efficiency.

[0063] This invention constructs a predictive dynamic electronic fence, significantly improving the safety of live-line work. Combining the Kalman prediction algorithm, this invention predicts the landing point of a laser spot at the next moment. Once the predicted landing point intrudes into the safety threshold of the transmission line, the system immediately cuts off the laser output at the hardware level. This protection mechanism effectively blocks the risk of accidental damage to the transmission line due to sudden target detachment or gimbal malfunction, meeting the extremely high safety requirements of smart grid operation and maintenance for the equipment itself.

[0064] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0065] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0066] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0067] Figure 1 This is a schematic diagram of the UAV-borne laser automatic obstacle clearing, tracking, and aiming system in an embodiment of the present invention;

[0068] Figure 2 This is a flowchart of the automatic obstacle clearing, tracking, and aiming method for an unmanned aerial vehicle (UAV)-borne laser in an embodiment of the present invention. Detailed Implementation

[0069] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0070] This embodiment details a hardware architecture scheme for an unmanned aerial vehicle (UAV)-borne laser-based automatic obstacle clearing, tracking, and aiming system. This system forms the physical basis for implementing the automatic obstacle clearing, tracking, and aiming method described in this invention, and its design must meet the power industry's requirements for insulation, electromagnetic interference resistance, and high reliability during high-voltage line operations.

[0071] like Figure 1 As shown, the overall system architecture mainly consists of four parts: the flight platform subsystem, the precision electro-optical targeting pod subsystem, the edge computing and control subsystem, and the ground integrated command and control terminal. The subsystems exchange information via a high-speed, low-latency airborne bus and wireless data links.

[0072] The flight platform subsystem, serving as the aerial carrier of the mission payload, preferably employs a large-wheelbase industrial-grade UAV with a hexacopter or octacopter configuration. The fuselage structure is constructed from a single piece of carbon fiber composite material to balance structural strength and lightweight requirements. To resist complex electromagnetic interference around high-voltage lines, the flight control unit (FCU) features a triple-redundant design and incorporates an anti-magnetic-interference industrial-grade compass and IMU (Inertial Measurement Unit). The positioning module utilizes dual-antenna RTK-GNSS (Real-time Dynamic Carrier Phase Differential) technology, receiving differential signals from ground base stations to achieve centimeter-level three-dimensional hovering positioning accuracy, providing a stable physical reference for subsequent laser targeting. The power system uses a high-energy-density solid-state lithium battery pack, coupled with an FOC (Field-Oriented Control) ESC and a high-efficiency brushless motor, ensuring sufficient effective operating endurance even when carrying a complete obstacle removal system (total weight approximately 5-10 kg).

[0073] The precision electro-optical targeting pod subsystem is mounted on the belly of the UAV via a quick-release shock-absorbing interface. This pod employs a three-axis high-precision stabilization gimbal architecture, providing rotational capabilities in three degrees of freedom: yaw, pitch, and roll. The gimbal integrates a high-torque-density direct-drive brushless motor and a high-resolution (better than 20-bit) absolute encoder, along with a built-in fiber optic gyroscope, achieving image stabilization accuracy better than 0.005 degrees and effectively isolating high-frequency vibrations from the UAV fuselage. Various sensors and actuators are precisely coaxially mounted on the gimbal's internal support structure.

[0074] The visual acquisition equipment preferably employs a dual-optical-path combination of visible light and infrared. The visible light channel uses a 1 / 1.8-inch large-area CMOS sensor, coupled with an optical zoom lens of 30x or more, capable of outputting high-definition real-time video streams at 1080P or 4K resolution for capturing fine textures of distant targets. The infrared channel uses an uncooled vanadium oxide focal plane array detector to detect the temperature field distribution of the target, particularly for monitoring the ablation state of obstacles and temperature changes of background conductors during laser heating.

[0075] The laser emitting device serves as the energy source for obstacle clearance missions, employing an integrated, packaged fiber-coupled semiconductor laser or fiber laser. This laser features dual modes: measurement and strike. In measurement mode, it emits low-power pulsed laser light that is safe for the human eye; in strike mode, it outputs high-energy continuous-wave (CW) laser light with wavelengths between 900 nm and 1080 nm. This wavelength range was chosen based on analysis of the spectral absorption characteristics of common debris materials such as polyethylene, polypropylene, and nylon. The laser's output port is equipped with an electrically operated zoom collimation mechanism, allowing the onboard processor to adjust the divergence angle of the laser beam in real time according to the target distance, thereby controlling the power density reaching the target surface.

[0076] The laser ranging module employs a 905nm or 1550nm wavelength Time-of-Flight (ToF) lidar. Its optical axis undergoes rigorous parallelism calibration with the visual acquisition device before leaving the factory. The module's measurement frequency is no less than 100Hz, with a measurement accuracy better than ±5cm, enabling real-time feedback of the precise slant distance of obstacles relative to the pod.

[0077] The edge computing and control subsystem is directly integrated inside the pod. This subsystem adopts an embedded heterogeneous computing architecture, such as a combination of FPGA+GPU+ARM. The FPGA is responsible for pixel-level pipeline processing of the front-end video stream (such as noise reduction and synchronization); the GPU is responsible for running the deep learning object detection network; and the ARM processor is responsible for running the real-time operating system, state machine logic, and Kalman filter tracking algorithm.

[0078] like Figure 2 As shown in the figure, this embodiment describes in detail how the system detects the target from the complex environmental background and achieves initial alignment after flying to the work area.

[0079] When the drone inspects a suspected fault location, the visual acquisition equipment starts working at full speed to acquire real-time video streams of the power transmission line corridor. The edge computing subsystem first preprocesses the video stream, including histogram equalization to enhance contrast and Gaussian filtering to remove sensor noise. Subsequently, the system loads a pre-trained deep convolutional neural network model (such as an improved YOLO series or Faster R-CNN). This model uses a massive dataset of power line scene data during training, covering typical hanging object shapes (such as kites, balloons, plastic sheeting, and dust nets) under different lighting conditions, angles, and backgrounds. The model can output the bounding boxes of obstacles in the image and their confidence scores in real time.

[0080] Once a target with a confidence level exceeding a preset threshold (e.g., 0.85) is identified, the system locks onto the target and calculates the geometric center coordinates of the bounding box as the pixel center position of the obstacle target in the image coordinate system. Simultaneously, the system calls the laser ranging module to obtain the target distance.

[0081] At this point, the system enters the coarse aiming phase. Due to the randomness of the UAV's hovering position, the target may initially be located in the edge region of the image. Simply relying on gimbal rotation may trigger mechanical limits or restrict the field of view. Therefore, this embodiment adopts a hierarchical control strategy of UAV-drone collaboration. The system calculates the deviation vector between the pixel center position and the image field of view center (optical axis center). This deviation vector is decomposed into two components: a low-frequency, large-amplitude offset component and a high-frequency, small-amplitude jitter component. The processor converts the low-frequency component into a flight position control signal for the UAV (forward, backward, left, and right translation speed), driving the UAV to slowly translate so that the target is roughly in front of it; at the same time, the high-frequency component and residual deviation are converted into gimbal attitude control commands, driving the gimbal motor to rotate rapidly. This ensures that the laser optical axis can quickly lock onto the target and that the gimbal always operates in the central attitude region with optimal mechanical performance.

[0082] This embodiment details the core technical feature of the present invention: the adaptive tracking step based on ablation feedback. This is the key to solving the problem of existing technologies failing due to smoke interference during laser strikes.

[0083] When a high-energy laser irradiates an obstacle, the material heats up and vaporizes, producing large amounts of smoke (water vapor, volatile organic compound particles, carbon black) as well as bright sparks or plasma. These byproducts can severely disrupt the stability of visual tracking, leading to target loss or laser misalignment. To address this challenge, this system introduces a real-time ablation state monitoring mechanism. During the tracking loop, the onboard processor executes a state analysis thread in parallel, calculating two key metrics in real time: the visual confidence factor and the ranging fluctuation factor.

[0084] Regarding the calculation of the visual confidence factor, the processor first extracts the texture features of the image within the bounding box region of the obstacle target. Specifically, it uses the Gray-Level Co-occurrence Matrix (GLCM) to calculate the image's entropy, energy, and contrast. Under normal smoke-free conditions, the target has rich texture details and a high entropy value. When smoke is generated, image details are blurred, the texture tends to be smoother, and the entropy value decreases. The processor calculates the rate of change of texture complexity between the current frame and the reference frame (the moment before laser emission). Simultaneously, the processor detects bright pixels within the region. By setting a brightness threshold (e.g., 255), it counts the number of saturated pixels and their growth rate. If bright pixels show an explosive growth, it is determined to be spark interference. It should be noted that, considering the slight displacement of the drone during hovering, to prevent background misalignment from interfering with texture analysis, the processor performs a global motion compensation step before extracting texture features. Specifically, it uses the ORB feature point matching algorithm to calculate the homography matrix of the current frame relative to the reference frame, performs perspective transformation correction on the current frame, and strictly aligns it with the reference frame in the pixel coordinate system.

[0085] In this embodiment, the texture complexity change rate is defined. Entropy decay rate: (in (This refers to the entropy value of the gray-level co-occurrence matrix); define the ranging fluctuation factor. Normalized standard deviation ratio: ,in To obtain the smaller value of the function, the system uses the following formula to calculate the visual confidence factor. : , in, The normalized growth rate of the number of highlighted pixels. and Environmental weighting coefficients (e.g., those taken under strong light conditions) ).

[0086] Furthermore, the ablation disturbance index The construction formula is: , in Assign weights (e.g., 0.6 and 0.4). Introduce squared terms. The design aims to nonlinearly amplify the effects of ranging fluctuations, enabling the system to quickly trigger a threshold when it detects smoke interference signals.

[0087] This embodiment describes in detail how to use the ablation perturbation index to dynamically adjust the tracking control algorithm in order to achieve anti-interference tracking.

[0088] This system employs an Extended Kalman Filter (EKF) as its core tracker, whose state vector includes the target's position, velocity, and acceleration in three-dimensional space. Specifically, the system uses a constant acceleration (CA) motion model to construct the state transition matrix. The state vector is defined as a nine-dimensional vector containing displacement, velocity, and acceleration components along the X, Y, and Z axes. The CA model is chosen because adrift objects typically exhibit nonlinear, variable-speed motion under the influence of wind or vaporization recoil. By including the acceleration term, even in inertial hold mode where visual feedback is interrupted, the system can still calculate the target's nonlinear oscillation trajectory in the short term based on the acceleration trend of the previous moment. In the Kalman filter, the measurement noise covariance matrix (R matrix) represents the system's level of confidence in the observed data.

[0089] This system has a pre-defined basic measurement noise covariance matrix, the value of which corresponds to the sensor's inherent noise level under smoke-free conditions. The specific mapping function is configured as an exponential growth model. .in, Let be the measurement noise covariance matrix at the current moment. The basic matrix for a smoke-free environment (determined by the inherent noise of the sensor). This is the sensitivity coefficient (preferred range 3.0 to 5.0). This function guarantees that when the ablation perturbation index... As the linear increase occurs, the system's confidence in visual measurements (i.e., ...) The weight of observations decreases exponentially, thus forcing the Kalman filter to automatically reduce the weight of observation updates, achieving a smooth switch to the motion prediction model.

[0090] When the laser begins ablation, producing a small amount of smoke and the ablation perturbation index rises slightly, the processor subtly increases the value of the R matrix through a mapping function. According to the Kalman filter gain calculation formula, as R increases, the Kalman gain decreases. This means the system recognizes that the visual sensor cannot clearly capture the target and actively reduces the weight of the current visual observation, relying more on the predictions of the motion model. The motion model is calculated based on the target's motion state (position, velocity, acceleration) before entering the smoke.

[0091] When combustion is intense, producing dense smoke that completely obscures visibility, causing the ablation disturbance index to rise sharply beyond a preset safety threshold, the mapping function will cause the R matrix to tend towards infinity (set as a maximum value in numerical calculations). At this point, the Kalman gain approaches zero. The system enters inertial hold mode. In this mode, the system completely cuts off the visual feedback loop and directly locks onto the gimbal's angular velocity command. Furthermore, in inertial hold mode (visual blind spot), the system executes a two-stage fuse protection mechanism in parallel:

[0092] 1. Burn-through sudden change fuse: The processor monitors the laser ranging value with the highest priority. If the ranging value changes abruptly during the smoke obscuring period (for example, the distance value increases by more than 2 meters instantaneously, indicating that the laser has burned through the obstacle or the target has fallen off), the system determines that it has missed the target and immediately cuts off the laser output to prevent accidental damage to the background conductor.

[0093] 2. Forced Exit After Timeout: If the duration of the inertial hold mode exceeds the preset safety threshold (e.g., 2.0 seconds) and the visual confidence factor has not recovered, the system will forcibly stop laser emission and reset the gimbal to prevent excessive cumulative error due to prolonged blind extrapolation.

[0094] Once the wind disperses the smoke and the visual features become clear again, the ablation perturbation index drops. The processor quickly reduces the R matrix, reintroduces the visual observation data, corrects the accumulated prediction errors, and achieves a seamless switch from blind following to precise following.

[0095] This embodiment provides a feedforward compensation scheme based on a physical model for thermal flutter, a microscopic physical phenomenon unique to laser obstacle removal.

[0096] In practical operations, especially when dealing with lightweight obstacles (such as balloon skins or ultra-thin plastic bags), when a high-power laser is focused on them, the high-speed jet generated by the vaporization of the material exerts a reaction force on the remaining solid material—a vaporization recoil force. Since the obstacles are usually suspended on flexible wires, this recoil force causes the target to experience high-frequency, irregular, and violent flutter. This flutter frequency often exceeds the closed-loop bandwidth of the vision tracking system, causing the laser spot to slip across the target surface and preventing energy concentration.

[0097] To address this issue, the system uses an image recognition algorithm to determine the material type of the obstacle (e.g., a black woven bag) during the coarse aiming phase. The processor internally stores a database of material thermophysical parameters, including the material's absorbance, latent heat of vaporization, density, and other parameters.

[0098] During laser emission, the processor reads the laser's output power in real time. Combining the spot area and material parameters, it estimates the mass flow rate of vaporized matter per unit time using the law of conservation of energy, and then calculates the instantaneous thrust (recoil force) acting on the target based on Newton's third law and the momentum theorem.

[0099] Because field operations cannot accurately obtain the microscopic physical properties of obstacles, and smoke obscures the real-time vibration phase, the processor performs the following engineering estimation steps based on empirical parameters and time synchronization:

[0100] The first step is recoil force and mass estimation. Based on the obstacle material classification determined by image recognition, the processor retrieves the corresponding empirical thrust coupling coefficient and empirical surface density parameters from a pre-set database. The instantaneous recoil force modulus is estimated by multiplying the laser emission power by the empirical thrust coupling coefficient; the theoretical mass is estimated by multiplying the visual projected area of ​​the obstacle by the empirical surface density parameters. It is particularly important to emphasize that, to avoid the dense smoke generated by subsequent laser ablation affecting the accuracy of the visual projected area calculation, this system implements an initial state locking strategy: the processor only extracts and stores the static visual projected area of ​​the obstacle and the aforementioned physical suspension length during the coarse aiming phase before the laser emission command is issued and before there is smoke in the field of view. Throughout the subsequent tracking and feedforward compensation process, regardless of whether the target in the real-time video stream is obscured by smoke, the stored initial static parameters are always used when estimating the theoretical mass, rather than using disturbed real-time observation values.

[0101] The second step is to establish a time-synchronized dynamic model. The system marks the moment when the laser emitting device turns on and emits light as time zero. A second-order damped oscillation function model starting from this time zero is constructed. The estimated recoil modulus is used as the step input signal, and the estimated theoretical mass is used as the inertial term. These are substituted into the function model to calculate the theoretical forced vibration trajectory of the obstacle target on the time axis.

[0102] The third step is to generate feedforward compensation commands. The processor generates a vibration suppression waveform that is out of phase and matches the amplitude of the theoretical forced vibration trajectory, and superimposes it into the gimbal's attitude control commands. This method utilizes the causal relationship between vibration and laser shock to effectively suppress the initial peak vibration through an open-loop time synchronization mechanism without requiring real-time visual feedback. Specifically, a compensation component with an out-of-phase phase to the theoretical oscillation trajectory is generated. Since the light emission moment is strictly set as the time zero point of the dynamic model (t=0), the laser shock force acts as a step input signal, and the phase of the initial vibration it excites is determined. Therefore, the system applies anti-phase feedforward control to this determined initial transient response, effectively suppressing the initial vibration amplitude, and then the control is gradually and smoothly transferred to the inertial holding mode.

[0103] According to the classification, the corresponding pre-calibrated thrust coupling coefficient and pre-calibrated surface density parameters are retrieved from the preset database.

[0104] The specific method for constructing and calibrating the preset database is as follows:

[0105] A laser damage effect testing platform was established in the laboratory, including a low-speed wind tunnel, a six-dimensional force / torque sensor (accuracy 0.001N), and a high-speed thermal imager.

[0106] 1. Sample preparation: Select typical hanging materials commonly found on power lines (such as black polyethylene agricultural film, polypropylene woven bags, and nylon kite cloth) and cut them into standard sample pieces.

[0107] 2. Offline calibration: The laser is controlled to irradiate the sample at stepped power (50W-500W), and the peak recoil force at the moment of vaporization is recorded using a force sensor. Through formula The thrust coupling coefficient was calculated; at the same time, the mass loss rate before and after ablation was measured using a precision balance to calibrate the areal density parameter.

[0108] 3. Database Generation: A mapping index is established between the above physical parameters and the visual characteristics of the materials (infrared emissivity, texture roughness), and stored in the non-volatile memory of the airborne processor. For unknown materials encountered during operation that are not in the database, the system defaults to calling a preset set of general conservative parameters (i.e., using the smallest thrust coupling coefficient in the database) to prevent overcompensation from causing oscillations.

[0109] This embodiment discloses typical parameters based on experimental calibration: for common black polyethylene (PE) agricultural films, the pre-calibrated areal density characteristic value is set to 20 g / m² to 30 g / m², and the pre-calibrated thrust coupling coefficient is set to 18 μN / W to 25 μN / W. It should be noted that, considering the dense smoke generated during laser obstacle clearance may obscure obstacles, making it impossible to accurately measure the real-time visual projection area, this system adopts an initial state locking strategy: during the coarse aiming stage before the laser emitting device is turned on (when the field of view is clear and smoke-free), the system uses an image recognition algorithm to lock the static visual projection area of ​​the obstacle target at the current viewpoint and stores it as a fixed constant. In subsequent tracking and flutter calculation processes, regardless of how much smoke interferes with vision, this initially locked visual projection area is multiplied by the preset areal density characteristic value to estimate the theoretical mass of the obstacle.

[0110] Next, the mass and recoil force are estimated.

[0111] The physical suspension length (denoted as ) from the attachment point to the geometric center of the obstacle target is calculated using image recognition. The specific calculation method follows the pinhole imaging projection model: First, the visual algorithm extracts the pixel distance between the obstacle's hanging point and its geometric center on the image plane. (Unit: pixels); Synchronously read the target slant range fed back by the laser ranging module. (Unit: meters) and the current focal length of the visual acquisition device. (Unit: pixels). Calculate the physical suspension length based on the principle of similar triangles. This step utilizes multi-sensor fusion data to solve the technical challenge of monocular vision being unable to directly measure physical dimensions when depth information is lacking, providing accurate geometric input for subsequent stiffness calculations.

[0112] To construct a solvable second-order dynamic model, this system introduces the small-angle oscillation equivalent assumption: the simple pendulum motion of the obstacle in the gravitational field is equivalent to the vibration of a spring-mass-damped system. Based on the restoring force of the pendulum... With Hooke's Law The correspondence is determined by the processor through formulas. Back-calculation of the equivalent stiffness coefficient of the system (in It is the acceleration due to gravity. (This refers to the theoretical mass estimated above). The recoil force generated by laser vaporization primarily excites the obstacle to undergo large-amplitude, low-frequency oscillations around the attachment point (this is the main reason why the target moves out of the field of view), followed by high-frequency flutter on the material surface. This step utilizes a dynamic model to predict and compensate for the trajectory of this large-amplitude, low-frequency oscillation, ensuring that the laser spot always covers the main path of the target's oscillation; while the residual high-frequency flutter component is passively suppressed by the geometric coverage area of ​​the laser spot itself (the spot diameter is usually larger than the flutter amplitude) and the mechanical inertia of the gimbal.

[0113] Finally, the dynamic equations are established. Setting the laser activation time to t=0, a second-order differential equation is constructed. (in For the laser power The calculated real-time vaporization recoil force, (Assuming pre-set air damping). The equation is numerically solved using the fourth-order Runge-Kutta method to obtain the theoretical oscillation trajectory.

[0114] The accuracy of physical parameters is crucial when constructing a dynamic model.

[0115] First, regarding the damping term Due to the irregular shape of the suspended object, this system introduces a dimensionless damping ratio. (For flexible thin film materials, the empirical value is taken as 0.05 to 0.15), and it is expressed by the formula. Calculate the equivalent damping coefficient in real time.

[0116] Second, regarding the conversion of control variables: dynamic equations The calculated value is the linear displacement of the target relative to its equilibrium position. (Unit: meters). The system needs to incorporate laser ranging values. Using the small angle approximation formula Convert it into the angular displacement compensation amount of the gimbal. (Unit: radians), and then superimposed into the attitude control command.

[0117] At this point, set the laser activation time to Taking the current static or slightly oscillating state as the initial condition, and using the laser vaporization recoil force... Given a step input, solving this equation yields the theoretical trajectory.

[0118] This embodiment elaborates on the system's safety baseline logic, particularly how to prevent lasers from accidentally damaging power transmission lines.

[0119] In laser obstacle removal operations, the most serious risk is the laser accidentally damaging power lines, leading to a decrease in their strength or even breakage. This system runs a high-priority power line feature extraction algorithm in parallel during the image processing flow. This algorithm utilizes edge detection operators (such as Canny or Sobel) and the Hough Transform to extract linear features of power transmission lines, ground wires, and optical cables in the background of the video stream in real time, and constructs their mathematical equations.

[0120] In each frame of the image, the processor calculates the Euclidean distance between the edge contour of the obstacle target (i.e., the laser strike point) and the nearest wire pixel. To convert the pixel distance in the image plane to the physical distance in real space, the processor performs the following projection transformation steps:

[0121] Read the current focal length (f) of the visual acquisition device and the target slant range (D) fed back by the laser ranging module. Based on the pinhole imaging principle, use the formula... Calculate the actual spatial distance, where The pixel distance on the image. This refers to the camera's focal length (in pixels). Through this projection transformation, the two-dimensional measurement of the image plane is mapped to a three-dimensional physical measurement. The system compares this actual spatial distance with a preset physical safety threshold (e.g., 0.05 meters), rather than directly using pixel values, thus eliminating the impact of changes in target distance on safety assessment. Furthermore, by combining the gimbal's motion state and utilizing the predictive function of a Kalman filter, the system predicts the laser spot's landing point at the next moment (e.g., 200 milliseconds in the future).

[0122] The system sets a strict electronic fence threshold (safety protection threshold). If the distance between the current spot position or the predicted landing point and the conductor is less than this threshold (e.g., less than 30 pixels in the image, equivalent to approximately 5 centimeters in physical distance), the underlying hardware interlock circuit will immediately activate, cutting off the power supply to the laser pump source. This protection is hardware-level, independent of the upper-level software logic, and remains effective even if the operating system crashes or the algorithm malfunctions.

[0123] To achieve system-level safety redundancy, the aforementioned protection mechanism consists of two layers: the software layer uses a Kalman prediction algorithm to predict the future spot's landing point, and slows down the gimbal in advance if there is an intrusion risk; the hardware layer employs an FPGA logic unit independent of the operating system. The FPGA directly reads the coordinate data stream from the vision sensor in parallel. Once the coordinate value of the current spot or obstacle edge reaches the preset electronic fence threshold, the FPGA directly pulls down the enable signal of the laser drive circuit through a physical pin, achieving a hardware-level forced power-off. This dual-redundancy design avoids the risk of accidental injury caused by software algorithm crashes or delays. This FPGA logic unit runs independently of the ARM embedded operating system, and the MIPI data stream from the vision sensor is input to the FPGA in parallel through a hardware splitter. Even if the upper-layer operating system crashes due to an algorithm infinite loop or memory overflow, the FPGA can still respond to the electronic fence boundary crossing risk within 1 millisecond based on pure hardware logic, achieving single-fault safety protection at the medical device level.

[0124] This embodiment describes the intelligent operation strategy of the system when facing severe weather conditions (such as no wind and high humidity).

[0125] In some cases, the smoke generated by the laser is not easy to dissipate, and continuous irradiation will only heat the smoke (producing a plasma shielding effect), wasting energy and easily causing a large-scale open fire.

[0126] Therefore, in this embodiment, when the ablation disturbance index remains above the preset fusing threshold for a certain period of time (e.g., 1 second), the processor determines that the current environment is no longer suitable for continued operation. The system automatically generates a laser pause command, stops laser emission, but maintains gimbal tracking (inertial mode). It utilizes natural micro-airflow to wait for the smoke to dissipate. During this period, the vision system continuously monitors environmental clarity. Once the visual confidence factor recovers to the observable range and remains stable for more than the preset anti-shake delay (e.g., 500 milliseconds), the system determines that the line of sight has been restored, immediately generates a laser recovery command, and recalibrates the aiming point. This not only improves safety but also prevents overheating of the wires around the target through intermittent cooling.

[0127] For complex scenarios where there are multiple hanging objects (such as a string of balloons or a large area of ​​damaged shade netting) within the same distance, this system has intelligent planning capabilities.

[0128] During the initial scanning phase, the system identifies all obstacle targets within its field of view. The processor performs a risk assessment on each target: calculating its distance from critical hardware such as insulator strings and vibration dampers; the closer the distance, the higher the risk level; and calculating its visual projection area; the smaller the area, the more likely it is to be a stress point or knot, and cutting it could lead to the entire structure falling off, thus maximizing efficiency.

[0129] The system generates a strike queue based on a principle of prioritizing risk levels while also considering clearance efficiency. Once the first target in the queue is cleared (visual confirmation of disappearance and a sudden change in ranging value), the system automatically drives the gimbal to quickly slew towards the next target using the 3D coordinates of the next target stored in the memory module, eliminating the need for manual re-searching. This reduces the workload of ground operators and improves operational efficiency.

[0130] To further illustrate the control accuracy of the system, this embodiment provides additional details on the frequency domain decoupling control between the UAV and the gimbal.

[0131] In actual flight, the maneuverability of a UAV is far lower than that of a gimbal. To achieve smooth tracking, the system must adaptively allocate control inputs. The system first calculates the total deviation vector between the obstacle pixel center position and the image field of view center. The system introduces a low-pass filter and a high-pass filter. The cutoff frequencies of the low-pass and high-pass filters are set to complementary values, typically between 0.1 Hz and 0.5 Hz (e.g., 0.2 Hz). Motion components below this frequency are considered large-scale drifts and are responded to by the UAV; motion components above this frequency are considered rapid jitters and are responded to by the onboard gimbal. This parameter setting is based on the physical bandwidth limitations of the UAV's large inertial motion. The total deviation vector is passed through the low-pass filter to extract the low-frequency, large-amplitude offset component. This component represents the overall orientation change of the target relative to the UAV, usually caused by wind-induced large-scale swaying of the target or the UAV's own drift. The system uses this low-frequency component to generate UAV flight control signals, driving the UAV to slowly translate, always attempting to keep the target in a general area directly in front of the nose. Simultaneously, the total deviation vector is filtered through a high-pass filter to extract a high-frequency, small-amplitude jitter component. This component represents the target's rapid tremors or the drone's vibration. The system uses this component to generate gimbal attitude control commands, driving the high-response gimbal motors to perform precise rotations. This avoids the energy consumption and instability caused by frequent drone maneuvers, while fully utilizing the gimbal's high-frequency response characteristics and eliminating aiming errors caused by rapid jitter.

[0132] This embodiment describes the system from a software architecture perspective.

[0133] The software architecture of the airborne edge computing unit adopts a layered and modular design. The bottom layer is the Hardware Abstraction Layer (HAL), which is responsible for parsing communication protocols with cameras, LiDAR, gimbal motor drivers, and flight control units. The middle layer is the core algorithm library, which includes computer vision (CV) libraries, digital signal processing (DSP) libraries, and motion control libraries. The top layer is the business logic layer, which is responsible for state machine scheduling and task management.

[0134] Raw data output from the visible light camera enters the FPGA via the MIPI interface. The FPGA's ISP pipeline performs de-mosaicing, automatic exposure (AE) statistics, automatic white balance (AWB), and highlight suppression (HLC). The processed YUV image is directly written to shared memory via the PCIe bus. The ARM processor's vision thread reads the image from shared memory, scales and normalizes it, and inputs it to the NPU (Neural Processing Unit) for inference. The inference result is fused with the state vector of the Kalman filter.

[0135] The system has a robust watchdog mechanism. If the visual tracking thread experiences a deadlock due to data anomalies for more than a preset time, the watchdog will trigger an interrupt, forcibly switching control to manual remote control mode and alerting the operator to take over. Furthermore, if the laser temperature sensor detects that the core module temperature exceeds a threshold, the thermal management subroutine will proactively reduce laser power or pause light output and run the cooling fan at full speed.

[0136] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for automatic obstacle clearing tracking and aiming using an unmanned aerial vehicle (UAV)-borne laser, applied to an obstacle clearing system comprising a UAV flight platform, an airborne gimbal, a visual acquisition device, and a laser emitting device, characterized in that... The method includes the following steps: The visual acquisition device acquires real-time monitoring video streams of the power transmission line channel, and controls the laser emitting device to emit ranging lasers to obtain distance information of obstacle targets; Image detection is performed on the real-time monitoring video stream to identify hanging obstacles on the transmission line, and the pixel center position of the obstacle in the image coordinate system is calculated. Based on the pixel center position and the distance information, a cooperative control command is generated to drive the UAV flight platform and the airborne gimbal to move, so that the laser aiming optical axis is aligned with the obstacle target; The method also includes an adaptive tracking step based on ablation feedback: During the process of controlling the laser emitting device to emit high-energy lasers for obstacle clearing, the ablation state of the obstacle target area is monitored in real time, and the visual confidence factor characterizing the visual observation quality and the distance measurement fluctuation factor characterizing the distance measurement stability are calculated. An ablation disturbance index is constructed based on the visual confidence factor and the ranging fluctuation factor. Based on the ablation perturbation index, the trust weights of visual observations and motion model predictions in the tracking control algorithm are dynamically adjusted; as the ablation perturbation index increases, the trust weight of the visual observations is reduced. When the ablation disturbance index exceeds a preset threshold, the airborne gimbal is controlled to enter the inertial hold mode, and the motion trend of the airborne gimbal is extrapolated using a motion model until the ablation disturbance index returns to the normal range.

2. The automatic obstacle clearing, tracking, and aiming method for an unmanned aerial vehicle (UAV) with a laser according to claim 1, characterized in that, The trust weights of visual observations and motion model predictions in the tracking control algorithm based on the ablation perturbation index include: A Kalman filter is used as the tracking control algorithm. Establish the mapping relationship between the ablation perturbation index and the Kalman filter measurement noise covariance matrix; As the ablation perturbation index increases, the value of the measurement noise covariance matrix increases according to a preset growth function; The Kalman gain is updated using the adjusted measurement noise covariance matrix, thereby reducing the dependence of system state updates on visual observations.

3. The automatic obstacle clearing, tracking, and aiming method for an unmanned aerial vehicle (UAV) with a laser as described in claim 1, characterized in that, The computational visual confidence factor includes: Extract image texture features within the bounding box region of the obstacle target and calculate the rate of change of texture complexity between the current frame and the previous frame; The number of bright pixels within the bounding box area is counted. If the growth rate of the number of bright pixels exceeds a preset threshold, it is determined that spark interference has occurred. The visual confidence factor is calculated by combining the texture complexity change rate and the number of highlighted pixels using a normalization function.

4. The automatic obstacle clearing, tracking, and aiming method for an unmanned aerial vehicle (UAV) with a laser as described in claim 1, characterized in that, The calculation of the ranging fluctuation factor includes: Establish a sliding time window for laser ranging data and collect distance values ​​from N consecutive sampling points; N is a positive integer. Calculate the standard deviation of the N sampling points and compare the standard deviation with a preset system noise benchmark value; If the standard deviation is greater than the system noise reference value, the ratio of the excess amplitude is determined as the ranging fluctuation factor.

5. The automatic obstacle clearing, tracking, and aiming method for an unmanned aerial vehicle (UAV) with a laser according to claim 1, characterized in that, The method also includes a feedforward compensation step based on thermally induced flutter: The current output power of the laser emitting device is obtained, and the classification of the obstacle target is determined using image recognition. According to the classification, the corresponding pre-calibrated thrust coupling coefficient and pre-calibrated surface density parameters are retrieved from the preset database; The vaporization recoil force is estimated based on the output power and the pre-calibrated thrust coupling coefficient, and the theoretical mass is estimated based on the visual projection area and the pre-calibrated surface density parameter. The physical suspension length from the hanging point to the geometric center of the obstacle target is calculated using image recognition, the natural oscillation frequency of the obstacle target is estimated based on the principle of the simple pendulum model, and the equivalent stiffness coefficient is calculated in reverse by combining the theoretical mass. The moment when the laser emitting device turns on to emit light is set as time zero. The theoretical oscillation trajectory of the obstacle target under the action of the vaporization recoil force is calculated using a dynamic model that includes the vaporization recoil force, theoretical mass and equivalent stiffness coefficient. A compensation component with the opposite phase to the theoretical oscillation trajectory is generated and superimposed on the gimbal attitude control command.

6. The automatic obstacle clearing, tracking, and aiming method for an unmanned aerial vehicle (UAV) with a laser according to claim 1, characterized in that, The method further includes an intermittent pulse modulation step: When the ablation disturbance index exceeds the preset safety fuse threshold, a laser pause command is generated to temporarily block the energy output of the laser emitting device. Image data is continuously acquired during the period when laser output is stopped, and the recovery of the visual confidence factor is continuously monitored; When the visual confidence factor rises back to the observable range and lasts for more than the preset anti-shake delay, a laser recovery command is generated, and the aiming point is recalibrated according to the current pixel center position.

7. The automatic obstacle clearing, tracking, and aiming method for an unmanned aerial vehicle (UAV) with a laser according to claim 1, characterized in that, The method includes a decoupled control step between the UAV position and the gimbal attitude: Calculate the deviation vector between the pixel center position and the image field of view center; The deviation vector is decomposed into a low-frequency, large-amplitude offset component and a high-frequency, small-amplitude jitter component. The large offset component is used to generate a UAV flight control signal, which drives the UAV flight platform to perform position translation to maintain coarse alignment. The gimbal attitude control command is generated using the small-amplitude jitter component, which drives the airborne gimbal to perform fine rotation to eliminate residual deviations.

8. The automatic obstacle clearing, tracking, and aiming method for an unmanned aerial vehicle (UAV) with a laser according to claim 1, characterized in that, The step of performing image detection on the real-time monitoring video stream also includes a power transmission line protection mechanism: While identifying the obstacle target, linear features of the power transmission lines in the background are extracted; Calculate the minimum pixel distance between the edge contour of the obstacle target and the power transmission line; Predict the location of the laser spot at the next moment using a prediction algorithm; If the distance between the landing point and the power transmission line is less than a preset safety protection threshold, the triggering permission of the laser emitting device is locked.

9. The automatic obstacle clearing, tracking, and aiming method for an unmanned aerial vehicle (UAV) equipped with a laser according to any one of claims 1 to 8, characterized in that, The method also includes a multi-objective intelligent sorting step: When the image detection results contain multiple hanging obstacles, the minimum distance between each obstacle and the key hardware of the transmission line is calculated as the risk level. Prioritize locking onto obstacles with the highest risk level and the smallest visual projection area at the attachment point location; After clearing the current obstacle, the memory module automatically retrieves the position information of the next highest priority obstacle and drives the airborne gimbal to turn quickly.

10. An unmanned aerial vehicle (UAV) laser-based automatic obstacle clearing, tracking, and aiming system, characterized in that: include: The perception module, including a visual acquisition device and a laser ranging module, is used to collect environmental data; The image processing unit is used to perform image detection on real-time monitoring video streams, identify obstacle targets, and calculate pixel center positions. The ablation state analysis unit is used to monitor the ablation state of obstacles and targets in real time during laser strikes, calculate the visual confidence factor and ranging fluctuation factor, and construct the ablation disturbance index. An adaptive controller is used to dynamically adjust the ratio of visual feedback weight to motion prediction weight in the tracking control algorithm based on the ablation disturbance index, and generate gimbal attitude control commands. The actuators, including the drone flight platform and airborne gimbal, are used to respond to control commands to achieve tracking and aiming; When the ablation disturbance index exceeds a threshold, the adaptive controller enters an inertial hold mode to resist smoke interference.

Images